Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music

• URL: http://arxiv.org/abs/2604.10905v1
• Date: Mon, 13 Apr 2026 02:11:56 GMT
• Title: Audio Flamingo Next: Next-Generation Open Audio-Language Models for Speech, Sound, and Music
• Abstract summary: AF-Next is the next-generation and most capable large audio-language model in the Audio Flamingo series.<n>It is designed to advance understanding and reasoning over speech, environmental sounds and music.<n> AF-Next is trained using a curriculum-based strategy spanning pre-training, mid-training and post-training stages.<n>In addition to all data, code and methods, we open-source 3 variants of AF-Next, including AF-Next-Instruct, AF-Next-Think and AF-Next-Captioner.
• Score: 112.20887496511165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Audio Flamingo Next (AF-Next), the next-generation and most capable large audio-language model in the Audio Flamingo series, designed to advance understanding and reasoning over speech, environmental sounds and music. Compared to Audio Flamingo 3, AF-Next introduces: (i) a stronger foundational audio-language model that significantly improves accuracy across diverse audio understanding tasks; (ii) scalable strategies for constructing large-scale audio understanding and reasoning data beyond existing academic benchmarks; (iii) support for long and complex audio inputs up to 30 minutes; and (iv) Temporal Audio Chain-of-Thought, a new reasoning paradigm that explicitly grounds intermediate reasoning steps to timestamps in long audio, enabling fine-grained temporal alignment and improved interpretability. To enable these capabilities, we first conduct a systematic analysis of Audio Flamingo 3 to identify key gaps in audio understanding and reasoning. We then curate and scale new large-scale datasets totaling over 1 million hours to address these limitations and expand the existing AudioSkills-XL, LongAudio-XL, AF-Think and AF-Chat datasets. AF-Next is trained using a curriculum-based strategy spanning pre-training, mid-training and post-training stages. Extensive experiments across 20 audio understanding and reasoning benchmarks, including challenging long-audio tasks, show that AF-Next outperforms similarly sized open models by large margins and remains highly competitive with and sometimes surpasses, much larger open-weight and closed models. Beyond benchmark performance, AF-Next exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability. In addition to all data, code and methods, we open-source 3 variants of AF-Next, including AF-Next-Instruct, AF-Next-Think and AF-Next-Captioner.

Related papers

• Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos [105.65597736594707]
  We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM)<n>Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo is designed for understanding and reasoning over long and complex real-world (audio-visual) videos.
  arXiv  Detail & Related papers  (2026-07-17T16:41:15Z)
• AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models [42.62457130960257]
  Large Audio-Language Models (LALMs) have shown strong performance on a wide range of audio understanding tasks, yet they still struggle with complex audio reasoning.<n>Existing audio-language datasets often contain substantial redundancy, where many samples are highly similar in acoustic content.<n>We propose a redundancy-aware data construction pipeline for building reasoning-oriented supervision for LALMs.
  arXiv  Detail & Related papers  (2026-06-12T16:09:49Z)
• AudioMarathon: A Comprehensive Benchmark for Long-Context Audio Understanding and Efficiency in Audio LLMs [53.248502396225724]
  AudioMarathon is a benchmark designed to evaluate both understanding and inference efficiency on long-form audio.<n>We evaluate state-of-the-art LALMs and observe clear performance drops as audio length grows.<n>The results show large gaps across current LALMs and highlight the need for better temporal reasoning.
  arXiv  Detail & Related papers  (2025-10-08T17:50:16Z)
• Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models [73.06287813212936]
  We present Audio Flamingo 3 (AF3), a fully open state-of-the-art (SOTA) large audio-language model that advances reasoning and understanding across speech, sound, and music.<n>AF3 introduces AF-Whisper, a unified audio encoder trained using a novel strategy for joint representation learning across all 3 modalities of speech, sound, and music.<n>It achieves new SOTA results on over 20+ (long) audio understanding and reasoning benchmarks, surpassing both open-weight and closed-source models trained on much larger datasets.
  arXiv  Detail & Related papers  (2025-07-10T19:40:21Z)
• Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities [72.91296768332163]
  We introduce Audio Flamingo 2 (AF2), an Audio-Language Model, and LongAudio, a dataset for training ALMs on long audio captioning and question-answering tasks.<n>AF2 achieves state-of-the-art performance with only a 3B parameter small language model, surpassing large open-source and proprietary models across over 20 benchmarks.<n>For the first time, we extend audio understanding to long audio segments (30 secs to 5 mins) and propose LongAudio, a large and novel dataset for training ALMs on long audio captioning and question-answering tasks.
  arXiv  Detail & Related papers  (2025-03-06T00:10:26Z)
• Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models [91.11904427660043]
  We introduce Audio-Reasoner, a large-scale audio language model for deep reasoning in audio tasks.<n>We train Audio-Reasoner on CoTA, enabling it to achieve great logical capabilities in audio reasoning.<n>Our findings stress the core of structured CoT training in advancing audio reasoning.
  arXiv  Detail & Related papers  (2025-03-04T06:18:34Z)
• Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities [37.02115473120654]
  Augmenting large language models (LLMs) to understand audio is critically important for diverse real-world applications. In this paper, we propose Audio Flamingo, a novel audio language model with 1) strong audio understanding abilities, 2) the ability to quickly adapt to unseen tasks via in-context learning and retrieval, and 3) strong multi-turn dialogue abilities.
  arXiv  Detail & Related papers  (2024-02-02T18:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.