DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models

• URL: http://arxiv.org/abs/2607.23944v1
• Date: Mon, 27 Jul 2026 02:40:49 GMT
• Title: DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models
• Abstract summary: This study proposes a Dual-Indicator Guided Contrastive Alignment (DICA)<n>DICA tracks two information-theoretic indicators during inference: Visual Attention Entropy (VAE) and Output Image Correlation (OIC)
• Score: 13.261187830239797
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human visual reasoning typically follows a coarse-to-fine attention process, starting from global scene understanding and gradually focusing on question-relevant regions. However, multimodal large language models may deviate from this pattern due to attention drift and the underutilization of visual evidence, which can lead to hallucinations. To mitigate these issues, this study proposes a Dual-Indicator Guided Contrastive Alignment (DICA), which tracks two information-theoretic indicators during inference: Visual Attention Entropy (VAE), which reflects the concentration of visual attention, and Output Image Correlation (OIC), which measures the dependence of generated outputs on the visual input. An abnormal increase in VAE or a decrease in OIC corresponds to different failure modes, which trigger targeted contrastive alignment to restore visual grounding. Experimental results across multiple benchmarks demonstrate that DICA consistently outperforms existing approaches and substantially reduces hallucinations, highlighting the effectiveness of indicator-driven intervention in improving multimodal inference reliability. The code is publicly available at https://github.com/BGWH123/DICA/.

Related papers

• What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes [1.9916820095972823]
  We introduce SAFE, a training-free decoding framework that contrasts visually-grounded and vision-ablated generation paths.<n>This signal serves dual roles: as a practical proxy for detecting visually-ungrounded tokens, and as the basis for decoding-time penalties.<n>We argue that designing contrastive probes exemplifies a broader mission: using models as instruments for scientific understanding.
  arXiv  Detail & Related papers  (2026-09-15T05:07:01Z)
• Disentangling Semantic Attention from Structural Bias in the Attention Manifold [61.68044165974505]
  Multimodal Large Language Models (MLLMs) exhibit disproportionate attention toward certain semantically uninformative visual tokens.<n>We introduce Saliency-guided Purification and Adaptive Redistribution (SPAR), a training-free, plug-and-play intervention.
  arXiv  Detail & Related papers  (2026-07-27T05:30:19Z)
• Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning [43.37915637521463]
  We propose a training-free adaptive visual gating method for unified multimodal reasoning.<n>AdaViG dynamically evaluates each triggered visual step at an early visual generation stage and aborts it when both signals are weak.
  arXiv  Detail & Related papers  (2026-07-10T22:03:01Z)
• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models [8.304027910542446]
  We investigate the internal attention dynamics of vision encoders in Large Vision-Language Models (LVLMs)<n>Our analysis reveals that hallucination behavior is particularly sensitive to tokens receiving low attention during the focus phase.<n>We propose a lightweight inference-time intervention that selectively suppresses such tokens during the focus phase.
  arXiv  Detail & Related papers  (2026-04-04T02:46:58Z)
• Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs [67.69730908817321]
  Internal Positive Attention Dynamics (PAD) in LVLMs naturally reveal semantically core visual regions under the distortions of attention sinks.<n>We propose Positive Attention Dynamics Enhancement (PADE), a training-free attention intervention that constructs a PAD map to identify semantically core visual regions.
  arXiv  Detail & Related papers  (2026-02-17T13:08:06Z)
• Attention-space Contrastive Guidance for Efficient Hallucination Mitigation in LVLMs [9.043999205886658]
  Hallucinations in large vision-language models often arise when language priors dominate over visual evidence.<n>We propose Contrastive Guidance (ACG), a single-pass mechanism that operates within self-attention layers to construct both vision-language and language-only attention paths.<n>ACG achieves state-of-the-art faithfulness and caption quality while significantly reducing computational cost.
  arXiv  Detail & Related papers  (2026-01-20T08:04:18Z)
• Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy [75.66913260900726]
  Reinforcement Learning with Verifiable Rewards has significantly advanced reasoning capabilities in Large Language Models.<n>Existing paradigms, driven by text-centric outcome rewards, encourage models to bypass visual perception.<n>We propose textbfThinking with Deltas, a framework driven by a textbfDifferential Visual Reasoning Policy.
  arXiv  Detail & Related papers  (2026-01-11T08:25:34Z)
• Mitigating Multimodal Hallucinations via Gradient-based Self-Reflection [49.26064449816502]
  We propose a Gradient-based Influence-Aware Constrained Decoding (GACD) method to address text-visual bias and co-occurrence bias.<n>GACD effectively reduces hallucinations and improves the visual grounding of MLLM outputs.
  arXiv  Detail & Related papers  (2025-09-03T08:13:52Z)
• IKOD: Mitigating Visual Attention Degradation in Large Vision-Language Models [20.036659182106806]
  We show that Large Vision-Language Models (LVLMs) exhibit a long-term bias where hallucinations increase as the sequence length grows.<n>We propose Image attention-guided Key-value merging cOllaborative Decoding (IKOD), a collaborative decoding strategy generating more image-focused sequences.
  arXiv  Detail & Related papers  (2025-08-05T14:05:15Z)
• Mitigating Hallucination for Large Vision Language Model by Inter-Modality Correlation Calibration Decoding [66.06337890279839]
  Large vision-language models (LVLMs) have shown remarkable capabilities in visual-language understanding for downstream multi-modal tasks.<n>LVLMs still suffer from generating hallucinations in complex generation tasks, leading to inconsistencies between visual inputs and generated content.<n>We propose an Inter-Modality Correlation Decoding (IMCCD) method to mitigate hallucinations in LVLMs in a training-free manner.
  arXiv  Detail & Related papers  (2025-01-03T17:56:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.