Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation

• URL: http://arxiv.org/abs/2604.01989v2
• Date: Fri, 03 Apr 2026 05:00:08 GMT
• Title: Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation
• Abstract summary: We propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention.<n>IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.
• Score: 50.51650162235191
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Like a body at rest that stays at rest, we find that visual attention in multimodal large language models (MLLMs) exhibits pronounced inertia, remaining largely static once settled during early decoding steps and failing to support the compositional understanding required for cognitive inference. While existing hallucination mitigation methods mainly target perceptual hallucinations concerning object existence or attributes, they remain inadequate for such cognitive hallucinations that require inter-object relational deduction. Through token-wise attention analysis, we identify this visual inertia as a key factor: attention to semantically critical regions remains persistently focused and fails to dynamically support relational inference. We thereby propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention. Specifically, IVE selects visual tokens that are dynamically emerging relative to historical attention trends while distinguishing tokens exhibiting inertial behavior. To further facilitate compositional inference, IVE introduces an inertia-aware penalty that discourages over-concentration and limits the persistence of attention within localized regions. Extensive experiments show that IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.

Related papers

• LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression [50.454633334898396]
  Large vision-language models (LVLMs) exhibit strong reasoning ability but suffer from visual forgetting during long-horizon decoding.<n>We propose LASER, a post-training framework that regulates both the visual attention trajectory and intra-visual token attention distribution during reasoning.<n>Experiments on eight benchmark datasets demonstrate that LASER consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-07-02T04:59:56Z)
• See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs [81.71384150441163]
  We propose Context-aware Attention Intervention (CAI) for large vision-language models (LVLMs)<n>CAI enforces a see only when needed principle via two-axis selectivity: where to look and when to intervene.<n> experiments across multiple LVLM backbones and benchmarks show that CAI achieves state-of-the-art hallucination mitigation.
  arXiv  Detail & Related papers  (2026-06-29T06:35:47Z)
• Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory [24.777344301465064]
  hallucinations are strongly associated with a human-like attention distraction phenomenon.<n>We propose an Attention-Focused Approach for Improved Image Perception, which corrects attention distraction via cross-head attention enrichment.
  arXiv  Detail & Related papers  (2026-05-23T14:36:06Z)
• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• Focus Matters: Phase-Aware Suppression for Hallucination in Vision-Language Models [8.304027910542446]
  We investigate the internal attention dynamics of vision encoders in Large Vision-Language Models (LVLMs)<n>Our analysis reveals that hallucination behavior is particularly sensitive to tokens receiving low attention during the focus phase.<n>We propose a lightweight inference-time intervention that selectively suppresses such tokens during the focus phase.
  arXiv  Detail & Related papers  (2026-04-04T02:46:58Z)
• Segmentation-Based Attention Entropy: Detecting and Mitigating Object Hallucinations in Large Vision-Language Models [9.388076929154673]
  Large Vision-Language Models (LVLMs) achieve strong performance on many multimodal tasks, but object hallucinations severely undermine their reliability.<n>Most existing studies focus on the text modality, attributing hallucinations to overly strong language priors and insufficient visual grounding.<n>We propose Attention Entropy (SAE), which leverages semantic segmentation to quantify visual attention uncertainty in an object-level semantic space.
  arXiv  Detail & Related papers  (2026-03-17T14:19:22Z)
• Revealing and Enhancing Core Visual Regions: Harnessing Internal Attention Dynamics for Hallucination Mitigation in LVLMs [67.69730908817321]
  Internal Positive Attention Dynamics (PAD) in LVLMs naturally reveal semantically core visual regions under the distortions of attention sinks.<n>We propose Positive Attention Dynamics Enhancement (PADE), a training-free attention intervention that constructs a PAD map to identify semantically core visual regions.
  arXiv  Detail & Related papers  (2026-02-17T13:08:06Z)
• Not All Tokens and Heads Are Equally Important: Dual-Level Attention Intervention for Hallucination Mitigation [46.3194503355054]
  Large vision-language models (LVLMs) have demonstrated impressive capabilities across diverse multimodal tasks.<n>They remain highly susceptible to visual hallucinations (VH), often producing confident but inaccurate descriptions.<n>We introduce VisFlow, a framework that alleviates hallucinations by directly modulating attention patterns during inference.
  arXiv  Detail & Related papers  (2025-06-14T19:10:22Z)
• Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence [69.86946427928511]
  We investigate the internal mechanisms driving hallucination in large vision-language models (LVLMs)<n>We introduce Vision-aware Head Divergence (VHD), a metric that quantifies the sensitivity of attention head outputs to visual context.<n>We propose Vision-aware Head Reinforcement (VHR), a training-free approach to mitigate hallucination by enhancing the role of vision-aware attention heads.
  arXiv  Detail & Related papers  (2024-12-18T15:29:30Z)
• Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models [69.79709804046325]
  We introduce R-Bench, a novel benchmark for evaluating Vision Relationship Hallucination. R-Bench features image-level questions that focus on the existence of relationships and instance-level questions that assess local visual comprehension. We identify three types of relationship co-occurrences that lead to hallucinations: relationship-relationship, subject-relationship, and relationship-object.
  arXiv  Detail & Related papers  (2024-06-24T08:42:42Z)
• Shifting Focus with HCEye: Exploring the Dynamics of Visual Highlighting and Cognitive Load on User Attention and Saliency Prediction [3.2873782624127834]
  This paper examines the joint impact of visual highlighting (permanent and dynamic) and dual-task-induced cognitive load on gaze behaviour. We show that state-of-the-art saliency models increase their performance when accounting for different cognitive loads.
  arXiv  Detail & Related papers  (2024-04-22T14:45:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.