How Contrastive Decoding Enhances Large Audio Language Models?

• URL: http://arxiv.org/abs/2603.09232v1
• Date: Tue, 10 Mar 2026 06:05:51 GMT
• Title: How Contrastive Decoding Enhances Large Audio Language Models?
• Abstract summary: Contrastive Decoding (CD) has proven effective at enhancing Large Audio Language Models (LALMs)<n>This study systematically evaluates four distinct CD strategies across diverse LALM architectures.<n>We identify Audio-Aware Decoding and Audio Contrastive Decoding as the most effective methods.
• Score: 49.26664187543512
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Contrastive Decoding (CD) has proven effective at enhancing Large Audio Language Models (LALMs), the underlying mechanisms driving its success and the comparative efficacy of different strategies remain unclear. This study systematically evaluates four distinct CD strategies across diverse LALM architectures. We identify Audio-Aware Decoding and Audio Contrastive Decoding as the most effective methods. However, their impact varies significantly by model. To explain this variability, we introduce a Transition Matrix framework to map error pattern shifts during inference. Our analysis demonstrates that CD reliably rectifies errors in which models falsely claim an absence of audio or resort to uncertainty-driven guessing. Conversely, it fails to correct flawed reasoning or confident misassertions. Ultimately, these findings provide a clear guideline for determining which LALM architectures are most suitable for CD enhancement based on their baseline error profiles.

Related papers

• An Evaluation Framework for Structured Audio Captions Validated by Controlled Perturbations [16.51678229294975]
  We propose a multi-axis evaluation framework tailored for structured audio descriptions.<n>We evaluate outputs across five axes: tag-sets, descriptions, logical reasoning, numeric measurements, and spectral profiles.<n>Our results demonstrate that this framework successfully distinguishes meaning-preserving paraphrases from genuine semantic and acoustic corruptions.
  arXiv  Detail & Related papers  (2026-07-23T15:26:52Z)
• LDGuid: A Framework for Robust Change Detection via Latent Difference Guidance [10.098765432618858]
  This paper proposes the Latent Difference Guidance (LDGuid) framework that explicitly learns and injects semantic differences into CD models.<n> Experimental results show that LDGuid enhances segmentation performance across all benchmarks.<n>Our findings suggest that semantic difference learning can drastically enhance the robustness of CD in remote sensing.
  arXiv  Detail & Related papers  (2026-05-15T03:45:36Z)
• Temporal Contrastive Decoding: A Training-Free Method for Large Audio-Language Models [56.91801348360746]
  Large audio-language models (LALMs) generalize across speech, sound, and music.<n> unified decoders can exhibit a emphtemporal smoothing bias<n>We propose emphTemporal Contrastive Decoding (TCD), a training-free decoding method for unified LALMs.
  arXiv  Detail & Related papers  (2026-04-16T02:30:41Z)
• Membership Inference Attacks against Large Audio Language Models [50.84901010528239]
  We present the first systematic Membership Inference Attack (MIA) evaluation of Large Audio Language Models (LALMs)<n>Using a multi-modal blind baseline based on textual, spectral, and prosodic features, we demonstrate that common speech datasets exhibit near-perfect train/test separability.<n>These findings establish a principled standard for auditing LALMs beyond spurious correlations.
  arXiv  Detail & Related papers  (2026-03-30T12:45:28Z)
• A SUPERB-Style Benchmark of Self-Supervised Speech Models for Audio Deepfake Detection [2.432576583937997]
  Spoof-SUPERB is a benchmark for audio deepfake detection.<n>We evaluate 20 SSL models spanning generative, discriminative, and spectrogram-based architectures.
  arXiv  Detail & Related papers  (2026-03-02T05:45:55Z)
• SWAP: Towards Copyright Auditing of Soft Prompts via Sequential Watermarking [58.475471437150674]
  We propose sequential watermarking for soft prompts (SWAP)<n>SWAP encodes watermarks through a specific order of defender-specified out-of-distribution classes.<n>Experiments on 11 datasets demonstrate SWAP's effectiveness, harmlessness, and robustness against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-11-05T13:48:48Z)
• Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs [63.82840470917859]
  We show that the decoding mechanism of dLLMs can be used as a powerful tool for model attribution.<n>We propose a novel information extraction scheme called the Directed Decoding Map (DDM), which captures structural relationships between decoding steps and better reveals model-specific behaviors.
  arXiv  Detail & Related papers  (2025-10-02T06:25:10Z)
• Decoding Uncertainty: The Impact of Decoding Strategies for Uncertainty Estimation in Large Language Models [58.198220611190884]
  We investigate the impact of decoding strategies on uncertainty estimation in Large Language Models (LLMs)<n>Our experiments show that Contrastive Search, which mitigates repetition, yields better uncertainty estimates on average across a range of preference-aligned LLMs.
  arXiv  Detail & Related papers  (2025-09-20T13:48:13Z)
• AVCD: Mitigating Hallucinations in Audio-Visual Large Language Models through Contrastive Decoding [38.71842806548495]
  We propose Audio-Visual Contrastive Decoding (AVCD) to model trimodal interactions and suppress hallucinations in large language models (MLLMs)<n>To improve efficiency, we introduce entropy-guided adaptive decoding, which skips unnecessary decoding steps based on the model's confidence in its predictions.
  arXiv  Detail & Related papers  (2025-05-27T08:13:57Z)
• FADEL: Uncertainty-aware Fake Audio Detection with Evidential Deep Learning [9.960675988638805]
  We propose a novel framework called fake audio detection with evidential learning (FADEL)<n>FADEL incorporates model uncertainty into its predictions, thereby leading to more robust performance in OOD scenarios.<n>We demonstrate the validity of uncertainty estimation by analyzing a strong correlation between average uncertainty and equal error rate (EER) across different spoofing algorithms.
  arXiv  Detail & Related papers  (2025-04-22T07:40:35Z)
• A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition [53.800937914403654]
  Advanced Audio-Visual Speech Recognition (AVSR) systems have been observed to be sensitive to missing video frames. While applying the dropout technique to the video modality enhances robustness to missing frames, it simultaneously results in a performance loss when dealing with complete data input. We propose a novel Multimodal Distribution Approximation with Knowledge Distillation (MDA-KD) framework to reduce over-reliance on the audio modality.
  arXiv  Detail & Related papers  (2024-03-07T06:06:55Z)
• Speculative Contrastive Decoding [55.378200871224074]
  Large language models(LLMs) exhibit exceptional performance in language tasks, yet their auto-regressive inference is limited due to high computational requirements and is sub-optimal due to the exposure bias. Inspired by speculative decoding and contrastive decoding, we introduce Speculative Contrastive Decoding(SCD), a straightforward yet powerful decoding approach.
  arXiv  Detail & Related papers  (2023-11-15T14:15:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.