TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models

• URL: http://arxiv.org/abs/2609.07286v1
• Date: Mon, 07 Sep 2026 09:54:52 GMT
• Title: TAD: Token-Adaptive Contrastive Decoding with Confidence-Guided Gating for Hallucination Mitigation in Large Audio-Language Models
• Abstract summary: Large audio-language models (LALMs) can hallucinate audio objects, answering "yes" to absent sound events.<n>We propose Token-Adaptive Decoding (TAD), a training-free strategy for hallucination mitigation.
• Score: 9.497925590553857
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large audio-language models (LALMs) can hallucinate audio objects, answering "yes" to absent sound events, thus undermining reliability in audio question answering. We propose Token-Adaptive Decoding (TAD), a training-free strategy for hallucination mitigation that grounds the initial yes/no decision by contrasting logits under real audio with a matched silent reference. TAD introduces a token-adaptive, confidence-guided gate that is decision-critical at the first decoding step and class-conditional on affirmative tokens, using the audio-silent margin to avoid overcorrection when evidence is weak or already sufficient. Experiments on AudioCaps-Hallucination show that, relative to Audio-Aware Decoding (AAD), a contrastive baseline with fixed contrast strength, TAD improves F1 for Qwen2 by 0.059 to 0.117 across Popular, Adversarial, and Random splits, and for Gemma by 0.025 to 0.064, while on Clotho-AQA it raises F1 from 0.810 to 0.816 on Qwen2 and remains comparable to AAD on Gemma.

Related papers

• Auditing Generative Audio Calls for Known-Task Audio-LLM Evaluation [8.345705067210861]
  For known closed-set tasks, access to acoustic evidence and the need to call a generative audio model are important.<n>We evaluate this distinction as a controlled call-decision problem.<n>For known-task endpoint claims, the relevant quantity is the marginal value of the generative call after transcript and encoder evidence have already been used.
  arXiv  Detail & Related papers  (2026-08-28T01:33:47Z)
• Audio-Zero: Label-Free Self-Evolution for Fine-Grained Audio Reasoning [44.01898427480094]
  Large Audio Language models (LALMs) have made rapid progress on acoustic understanding, yet they still struggle with fine-grained audio reasoning.<n>We introduce Audio-Zero, the first label-free self-evolution framework in the field of LALMs.<n>We show that Audio-Zero improves fine-grained audio reasoning while preserving broad audio understanding.
  arXiv  Detail & Related papers  (2026-07-22T14:03:25Z)
• Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition [58.25449304752214]
  We study whether explicit acoustic cues are grounded when the raw audio is already available.<n>We deriving six interpretable acoustic concept tokens from the standardised eGeMAPS paralinguistic feature set.<n>Aligned tokens improve unweighted average recall (UAR), whereas shuffled, conflicting, or corrupted tokens reduce performance.<n>We argue that token-only interventions provide a practical way to probe audio-grounded cue use, robustness, and interpretability in ALM-based affective computing.
  arXiv  Detail & Related papers  (2026-06-05T14:26:06Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• Don't Let the Video Speak: Audio-Contrastive Preference Optimization for Audio-Visual Language Models [57.29797562658234]
  Cross-modal hallucination is bottlenecked by Audio-Visual Language Models (AVLMs)<n>We propose Audio-Contrastive Preference Optimization (ACPO) to counteract this deeply ingrained visual dominance.<n>ACPO introduces an output-contrastive objective to penalize visual descriptions masquerading as audio facts, alongside an input-contrastive objective that swaps audio tracks to explicitly penalize generation invariant to the true auditory signal.
  arXiv  Detail & Related papers  (2026-04-15T17:51:28Z)
• When Eyes and Ears Disagree: Can MLLMs Discern Audio-Visual Confusion? [41.579901082251254]
  Experimental results reveal that Multimodal Large Language Models (MLLMs) struggle to discriminate non-existent audio due to visually dominated reasoning.<n>Motivated by this observation, we introduce RL-CoMM, a Reinforcement Learning-based Collaborative Multi-MLLM.<n> RL-CoMM includes two stages: 1) To alleviate visually dominated ambiguities, we introduce an external model, a Large Audio Language Model (LALM), as the reference model to generate audio-only reasoning.
  arXiv  Detail & Related papers  (2025-11-13T07:59:41Z)
• Adaptive vector steering: A training-free, layer-wise intervention for hallucination mitigation in large audio and multimodal models [45.88028371034407]
  We propose Adaptive Vector Steering to better grounds generation in audio content.<n>Experiments show consistent performance gains across two models and two benchmarks.<n>To the best of our knowledge, this is the first work to apply vector steering to mitigate hallucination in audio.
  arXiv  Detail & Related papers  (2025-10-14T08:52:18Z)
• Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding [54.82619273983179]
  Large Audio-Language Models (LALMs) can hallucinate what is presented in the audio.<n>We introduce Audio-Aware Decoding (AAD) to mitigate the hallucination of LALMs.<n>AAD uses contrastive decoding to compare the token prediction logits with and without the audio context.
  arXiv  Detail & Related papers  (2025-06-08T17:36:50Z)
• Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples [55.2480439325792]
  Recent advancements in audio-aware large language models (ALLMs) enable them to process and understand audio inputs.<n>These models often hallucinate non-existent sound events, reducing their reliability in real-world applications.<n>We propose LISTEN, a contrastive-like training method that enhances ALLMs' ability to distinguish between present and absent sounds.
  arXiv  Detail & Related papers  (2025-05-20T15:44:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.