From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning

• URL: http://arxiv.org/abs/2603.03825v1
• Date: Wed, 04 Mar 2026 08:22:27 GMT
• Title: From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning
• Abstract summary: We find that reasoning performance is strongly correlated with Visual Attention Score (VAS)<n>To validate its causal role, we design training-free interventions that directly modulate attention allocation during inference.<n>We propose Attention-Guided Visual Anchoring and Reflection, a comprehensive cold-start framework that integrates visual-anchored data synthesis, attention-guided objectives, and visual-anchored reward shaping.
• Score: 72.4876727619987
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The cold-start initialization stage plays a pivotal role in training Multimodal Large Reasoning Models (MLRMs), yet its mechanisms remain insufficiently understood. To analyze this stage, we introduce the Visual Attention Score (VAS), an attention-based metric that quantifies how much a model attends to visual tokens. We find that reasoning performance is strongly correlated with VAS (r=0.9616): models with higher VAS achieve substantially stronger multimodal reasoning. Surprisingly, multimodal cold-start fails to elevate VAS, resulting in attention distributions close to the base model, whereas text-only cold-start leads to a clear increase. We term this counter-intuitive phenomenon Lazy Attention Localization. To validate its causal role, we design training-free interventions that directly modulate attention allocation during inference, performance gains of 1$-$2% without any retraining. Building on these insights, we further propose Attention-Guided Visual Anchoring and Reflection (AVAR), a comprehensive cold-start framework that integrates visual-anchored data synthesis, attention-guided objectives, and visual-anchored reward shaping. Applied to Qwen2.5-VL-7B, AVAR achieves an average gain of 7.0% across 7 multimodal reasoning benchmarks. Ablation studies further confirm that each component of AVAR contributes step-wise to the overall gains. The code, data, and models are available at https://github.com/lrlbbzl/Qwen-AVAR.

Related papers

• Token-level Response-visual Attention Guidance for Multimodal LLMs Knowledge Distillation [36.9736177938003]
  We introduce Token-level Response-visual Attention Guidance (TRAG)<n>TRAG shifts the focus to response-to-vision signals and employs token-specific objectives.<n>Extensive experimental results on multiple benchmarks demonstrate that TRAG significantly outperforms prior distillation baselines.
  arXiv  Detail & Related papers  (2026-07-01T08:03:30Z)
• Visually-Guided Policy Optimization for Multimodal Reasoning [60.035908460318126]
  We propose Visually-Guided Policy Optimization (VGPO) to reinforce visual focus during policy optimization.<n> VGPO initially introduces a Visual Attention Compensation mechanism that leverages visual similarity to localize and amplify visual cues.<n>We show that VGPO achieves better visual activation and superior performance in mathematical multimodal reasoning and visual-dependent tasks.
  arXiv  Detail & Related papers  (2026-04-10T14:22:38Z)
• Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models [38.47027398567909]
  Perception-Grounded Policy Optimization (PGPO) is a novel fine-grained credit assignment framework that dynamically reshapes advantages at the token level.<n>We show that PGPO actively amplifies learning signals for visually-dependent tokens while suppressing gradient noise from linguistic priors.<n>Both theoretical and empirical analyses confirm that PGPO effectively reduces gradient variance, prevents training collapse, and acts as a potent regularizer for robust, perception-grounded multimodal reasoning.
  arXiv  Detail & Related papers  (2026-04-02T09:53:20Z)
• Counting Circuits: Mechanistic Interpretability of Visual Reasoning in Large Vision-Language Models [35.71430064413904]
  Counting serves as a powerful test of a Large Vision-Language Model's (LVLM's) reasoning.<n>We show that LVLMs display a human-like counting behavior, with precise performance on small numerosities and noisy estimation for larger quantities.<n>We propose a lightweight intervention strategy that exploits simple and abundantly available synthetic images to fine-tune arbitrary pretrained LVLMs exclusively on counting.
  arXiv  Detail & Related papers  (2026-03-19T06:10:10Z)
• ActionCodec: What Makes for Good Action Tokenizers [106.78093973045526]
  Vision-Language-Action (VLA) models have demonstrated superior instruction-following and training efficiency.<n>Central to this paradigm is action tokenization, yet its design has primarily focused on reconstruction fidelity.<n>We introduce textbfActionCodec, a high-performance action tokenizer that significantly enhances both training efficiency and VLA performance.
  arXiv  Detail & Related papers  (2026-02-17T07:07:15Z)
• Credit Where It is Due: Cross-Modality Connectivity Drives Precise Reinforcement Learning for MLLM Reasoning [11.021067780524348]
  We show how visual evidence is integrated during reasoning remains poorly understood.<n>We propose Anchor-Token Reinforcement Learning (AT-RL), a lightweight framework that selectively reinforces high-connectivity tokens.<n>Our work reveals that reasoning quality is governed not by token quantity but by the fidelity of cross-modal anchoring.
  arXiv  Detail & Related papers  (2026-02-12T00:20:54Z)
• ChainV: Atomic Visual Hints Make Multimodal Reasoning Shorter and Better [59.29940512530982]
  We propose ChainV, a framework that dynamically integrates visual hints into the reasoning process.<n>Our method significantly improves reasoning accuracy and efficiency, especially on math-intensive benchmarks.
  arXiv  Detail & Related papers  (2025-11-21T10:11:17Z)
• Spotlight on Token Perception for Multimodal Reinforcement Learning [65.97597482517425]
  Reinforcement Learning with Verifiable Rewards (RLVR) has advanced the reasoning capabilities of Large Vision-Language Models (LVLMs)<n>In this paper, we undertake a pioneering exploration of multimodal RLVR through the novel perspective of token perception.<n>We propose Visually-Perceptive Policy Optimization (VPPO), a novel policy gradient algorithm that explicitly leverages token perception to refine the learning signal.
  arXiv  Detail & Related papers  (2025-10-10T11:25:33Z)
• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models [63.69856480318313]
  AGILE formulates jigsaw solving as an interactive process, enabling the model to progressively engage with the environment.<n>We show that AGILE substantially boosts performance on jigsaw tasks of varying complexity.<n>We also demonstrate strong generalization across 9 general vision tasks, achieving an average improvement of 3.1%.
  arXiv  Detail & Related papers  (2025-10-01T17:58:05Z)
• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning [70.44416154144001]
  Recent strides in multimodal large language models (MLLMs) have significantly advanced their performance in many reasoning tasks.<n> Abstract Visual Reasoning (AVR) remains a critical challenge, primarily due to limitations in perceiving abstract graphics.<n>We propose VisuRiddles, a benchmark for PRS, featuring tasks meticulously constructed to assess models' reasoning capacities.<n>Second, we introduce the Perceptual Riddle Synthesizer (PRS), an automated framework for generating riddles with fine-grained perceptual descriptions.
  arXiv  Detail & Related papers  (2025-06-03T07:24:00Z)
• STAR: Stage-Wise Attention-Guided Token Reduction for Efficient Large Vision-Language Models Inference [3.9464481148889354]
  We propose STAR (Stage-wise Attention-guided token Reduction), a training-free, plug-and-play framework that approaches token pruning from a global perspective.<n>Instead of pruning at a single point, STAR performs attention-guided reduction in two complementary stages: an early-stage pruning based on visual self-attention to remove redundant low-level features, and a later-stage pruning guided by cross-modal attention to discard task-irrelevant tokens.
  arXiv  Detail & Related papers  (2025-05-18T10:44:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.