HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models

• URL: http://arxiv.org/abs/2604.19300v1
• Date: Tue, 21 Apr 2026 10:05:28 GMT
• Title: HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
• Abstract summary: We introduce HalluAudio, the first large-scale benchmark for evaluating hallucinations across speech, environmental sound, and music.<n>HalluAudio comprises over 5K human-verified QA pairs and spans diverse task types, including binary judgments, multi-choice reasoning, attribute verification, and open-ended QA.<n>Our results reveal significant deficiencies in acoustic grounding, temporal reasoning, and music attribute understanding, underscoring the need for reliable and robust LALMs.
• Score: 30.18524844766061
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Audio-Language Models (LALMs) have recently achieved strong performance across various audio-centric tasks. However, hallucination, where models generate responses that are semantically incorrect or acoustically unsupported, remains largely underexplored in the audio domain. Existing hallucination benchmarks mainly focus on text or vision, while the few audio-oriented studies are limited in scale, modality coverage, and diagnostic depth. We therefore introduce HalluAudio, the first large-scale benchmark for evaluating hallucinations across speech, environmental sound, and music. HalluAudio comprises over 5K human-verified QA pairs and spans diverse task types, including binary judgments, multi-choice reasoning, attribute verification, and open-ended QA. To systematically induce hallucinations, we design adversarial prompts and mixed-audio conditions. Beyond accuracy, our evaluation protocol measures hallucination rate, yes/no bias, error-type analysis, and refusal rate, enabling a fine-grained analysis of LALM failure modes. We benchmark a broad range of open-source and proprietary models, providing the first large-scale comparison across speech, sound, and music. Our results reveal significant deficiencies in acoustic grounding, temporal reasoning, and music attribute understanding, underscoring the need for reliable and robust LALMs.

Related papers

• MRMAD: A Multi-Round Multi-Audio Benchmark for Evaluating Acoustic Degradation Perception in Large Audio-Language Models [11.14810883033264]
  We introduce MRMAD, a benchmark for evaluating audio degradation perception and understanding in large audio-language models (LALMs)<n> MRMAD spans speech, music, and sound, and frames evaluation as multi-turn dialogues across multiple audio inputs.<n>We find that current models often recognize coarse content while failing to diagnose, compare, or reason about degradations reliably.
  arXiv  Detail & Related papers  (2026-08-23T06:23:31Z)
• SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Models [32.25458333197992]
  We show that speech content can induce hallucinations in audio-visual large-language models (LLMs)<n>Our benchmark diagnoses speech-vision hallucinations from two critical and complementary aspects: semantic and temporal.<n>Our work uncovers a new and fundamental limitation of current audio-visual LLMs and highlights the need for speech-grounded video comprehension.
  arXiv  Detail & Related papers  (2026-05-31T14:00:37Z)
• Exploring Audio Hallucination in Egocentric Video Understanding [66.1760617001607]
  State-of-the-art large audio-visual language models (AV-LLMs) can generate multimodal descriptions.<n>We show in this work that they are prone to audio hallucinations, often inferring sounds from visual cues that are visible but not heard.
  arXiv  Detail & Related papers  (2026-04-26T20:06:58Z)
• Noise-Aware In-Context Learning for Hallucination Mitigation in ALLMs [5.553031534100783]
  Auditory large language models (ALLMs) have demonstrated strong general capabilities in audio understanding and reasoning tasks.<n>We propose a plug-and-play Noise-Aware In-Context Learning ( NAICL) method to address hallucination issues.
  arXiv  Detail & Related papers  (2026-04-10T06:35:46Z)
• SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation [52.468945848774844]
  This paper addresses the need for automated systems capable of evaluating audio separation without human intervention.<n>The proposed evaluation metric, SAM Audio Judge (SAJ), is a multimodal fine-grained reference-free objective metric.<n>SAJ supports three audio domains (speech, music and general sound events) and three prompt inputs (text, visual and span), covering four different dimensions of evaluation.
  arXiv  Detail & Related papers  (2026-01-27T15:29:02Z)
• JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation [16.067014259345743]
  We evaluate leading vision-only, audio-only, and Omni-LLMs on our dataset.<n>Even the best-performing Omni-LLM achieves an average accuracy of only 62.6%, outperforming uni-modal baselines.
  arXiv  Detail & Related papers  (2025-12-14T17:23:21Z)
• Spatial Blind Spot: Auditory Motion Perception Deficits in Audio LLMs [39.209987830131816]
  Large Audio-Language Models (LALMs) have recently shown impressive progress in speech recognition, audio captioning, and auditory question answering.<n>Yet, whether these models can perceive dynamics, particularly the motion of sound sources, remains unclear.<n>We introduce AMPBench, the first benchmark explicitly designed to evaluate auditory motion understanding.
  arXiv  Detail & Related papers  (2025-11-17T11:45:41Z)
• Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding [54.82619273983179]
  Large Audio-Language Models (LALMs) can hallucinate what is presented in the audio.<n>We introduce Audio-Aware Decoding (AAD) to mitigate the hallucination of LALMs.<n>AAD uses contrastive decoding to compare the token prediction logits with and without the audio context.
  arXiv  Detail & Related papers  (2025-06-08T17:36:50Z)
• Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge [102.84031769492708]
  This task defines three QA subsets to test audio-language models on interactive question-answering over diverse acoustic scenes.<n>Preliminary results on the development set are compared, showing strong variation across models and subsets.<n>This challenge aims to advance the audio understanding and reasoning capabilities of audio-language models toward human-level acuity.
  arXiv  Detail & Related papers  (2025-05-12T09:04:16Z)
• AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models [27.430040932849018]
  We introduce AVHBench, the first comprehensive benchmark specifically designed to evaluate the perception and comprehension capabilities of audio-visual models.<n>Our results reveal that most existing audio-visual LLMs struggle with hallucinations caused by cross-interactions between modalities.<n>We demonstrate that simple training with our AVHBench improves robustness of audio-visual LLMs against hallucinations.
  arXiv  Detail & Related papers  (2024-10-23T23:36:06Z)
• Can Large Audio-Language Models Truly Hear? Tackling Hallucinations with Multi-Task Assessment and Stepwise Audio Reasoning [55.2480439325792]
  Large audio-language models (LALMs) have shown impressive capabilities in understanding and reasoning about audio and speech information.<n>These models still face challenges, including hallucinating non-existent sound events, misidentifying the order of sound events, and incorrectly attributing sound sources.
  arXiv  Detail & Related papers  (2024-10-21T15:55:27Z)
• Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models [56.776580717999806]
  Real-world applications often involve processing multiple audio streams simultaneously. We propose the first multi-audio evaluation benchmark that consists of 20 datasets from 11 multi-audio tasks. We propose a novel multi-audio-LLM (MALLM) to capture audio context among multiple similar audios.
  arXiv  Detail & Related papers  (2024-09-27T12:06:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.