Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models

• URL: http://arxiv.org/abs/2603.14184v1
• Date: Sun, 15 Mar 2026 02:21:05 GMT
• Title: Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models
• Abstract summary: Multimodal large language models (MLLMs) often suffer from perceptual impairments under extended reasoning modes.<n>We identify attention dispersion as the underlying cause: during multi-step reasoning, the model's visual attention becomes scattered and drifts away from question-relevant regions, effectively "losing focus" on the visual input.<n>We propose a training-free Visual Region-Guided Attention (VRGA) framework that selects visual heads based on an entropy-focus criterion and reweights their attention, effectively guiding the model to focus on question-relevant regions during reasoning.
• Score: 15.851502442699
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models (MLLMs) often suffer from perceptual impairments under extended reasoning modes, particularly in visual question answering (VQA) tasks. We identify attention dispersion as the underlying cause: during multi-step reasoning, the model's visual attention becomes scattered and drifts away from question-relevant regions, effectively "losing focus" on the visual input. To better understand this phenomenon, we analyze the attention maps of MLLMs and observe that reasoning prompts significantly reduce attention to regions critical for answering the question. We further find a strong correlation between the model's overall attention on image tokens and the spatial dispersiveness of its attention within the image. Leveraging this insight, we propose a training-free Visual Region-Guided Attention (VRGA) framework that selects visual heads based on an entropy-focus criterion and reweights their attention, effectively guiding the model to focus on question-relevant regions during reasoning. Extensive experiments on vision-language benchmarks demonstrate that our method effectively alleviates perceptual degradation, leading to improvements in visual grounding and reasoning accuracy while providing interpretable insights into how MLLMs process visual information.

Related papers

• LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression [50.454633334898396]
  Large vision-language models (LVLMs) exhibit strong reasoning ability but suffer from visual forgetting during long-horizon decoding.<n>We propose LASER, a post-training framework that regulates both the visual attention trajectory and intra-visual token attention distribution during reasoning.<n>Experiments on eight benchmark datasets demonstrate that LASER consistently outperforms strong baselines.
  arXiv  Detail & Related papers  (2026-07-02T04:59:56Z)
• Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory [24.777344301465064]
  hallucinations are strongly associated with a human-like attention distraction phenomenon.<n>We propose an Attention-Focused Approach for Improved Image Perception, which corrects attention distraction via cross-head attention enrichment.
  arXiv  Detail & Related papers  (2026-05-23T14:36:06Z)
• Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models [65.57985131861399]
  hallucinations are strongly correlated with cognitive bifurcation points that often exhibit high entropy states.<n>We propose V-STAR, a lightweight, holistic training paradigm designed to internalize visually aware reasoning capabilities.
  arXiv  Detail & Related papers  (2026-04-11T13:59:05Z)
• Attention at Rest Stays at Rest: Breaking Visual Inertia for Cognitive Hallucination Mitigation [50.51650162235191]
  We propose a training-free Inertia-aware Visual Excitation (IVE) method that breaks this inertial pattern by modeling cognitive inference as the dynamic responsiveness of visual attention.<n>IVE is effective across various base MLLMs and multiple hallucination benchmarks, particularly for cognitive hallucinations.
  arXiv  Detail & Related papers  (2026-04-02T12:51:07Z)
• Decoding the Pulse of Reasoning VLMs in Multi-Image Understanding Tasks [0.006813985320936554]
  We investigate a previously overlooked phenomenon: during chain-of-thought (CoT) generation, the text-to-image (T2I) attention of reasoning exhibits diffuse "pulses"<n>We propose PulseFocus, a training-free, inference-time method that structures CoT reasoning into interleaved plan/focus blocks with soft attention gating.
  arXiv  Detail & Related papers  (2026-03-04T23:34:39Z)
• Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs [55.61018839017648]
  Chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks.<n>Existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies.<n>We propose SAYO, a visual reasoning model trained with a reinforcement learning framework that introduces a region-level visual attention-based reward.
  arXiv  Detail & Related papers  (2026-02-09T03:33:23Z)
• Focusing by Contrastive Attention: Enhancing VLMs' Visual Reasoning [79.34909830834464]
  Vision-Language Models (VLMs) have demonstrated remarkable success across diverse visual tasks, yet their performance degrades in complex visual environments.<n>We show that visual complexity strongly correlates with attention entropy, negatively impacting reasoning performance.<n>We propose Contrastive Attention Refinement for Visual Enhancement (CARVE), a training-free method that extracts task-relevant visual signals through attention contrasting at the pixel level.
  arXiv  Detail & Related papers  (2025-09-08T09:20:04Z)
• Look-Back: Implicit Visual Re-focusing in MLLM Reasoning [15.478700750705643]
  Multimodal Large Language Models (MLLMs) have achieved remarkable progress in multimodal reasoning.<n>Current methods typically address this by explicitly injecting visual information to guide the reasoning process.<n>We introduce Look-Back, an implicit approach designed to guide MLLMs to look back" at visual information in a self-directed manner during reasoning.
  arXiv  Detail & Related papers  (2025-07-02T14:59:35Z)
• Exploring Implicit Visual Misunderstandings in Multimodal Large Language Models through Attention Analysis [21.869968563545736]
  We define implicit visual misunderstanding (IVM), where MLLMs provide correct answers without fully comprehending the visual input.<n>We introduce a scale-agnostic metric, textitattention accuracy, and a novel benchmark for quantifying IVMs.<n>We extend our approach to finer granularities and demonstrate its effectiveness in unimodal scenarios.
  arXiv  Detail & Related papers  (2025-05-15T17:52:40Z)
• Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas [69.56484419619919]
  We study the spatial reasoning challenge from the lens of mechanistic interpretability.<n>We observe that successful spatial reasoning correlates strongly with the model's ability to align its attention with actual object locations.<n>Motivated by these findings, we propose ADAPTVIS to sharpen the attention on highly relevant regions when confident.
  arXiv  Detail & Related papers  (2025-03-03T17:57:03Z)
• MINT: Mitigating Hallucinations in Large Vision-Language Models via Token Reduction [6.416957959150438]
  Hallucinations hinder the application of Large Vision-Language Models (LVLMs) in domains that require high reliability.<n>We propose MINT, a training-free decoding strategy, MItigating hallucinations via tokeN reducTion.<n>Our approach achieves a 4% improvement in mitigating hallucinations caused by distracted perception compared to original models.
  arXiv  Detail & Related papers  (2025-02-02T08:34:57Z)
• Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference [68.12511526813991]
  We provide a novel understanding of multi-head attention from a Bayesian perspective. We propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention. Experiments on various attention models and applications demonstrate that the proposed repulsive attention can improve the learned feature diversity.
  arXiv  Detail & Related papers  (2020-09-20T06:32:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.