The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning

• URL: http://arxiv.org/abs/2607.11436v2
• Date: Fri, 17 Jul 2026 15:34:16 GMT
• Title: The Ebb and Flow of Multimodal Focus: Scheduling Visual Relay Windows for Grounded VLM Reasoning
• Abstract summary: Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack.<n>We propose TRACE, a task-adaptive inference-time control framework with lightweight trained modules.<n>Across four open-weight VLM backbones and seven benchmarks, TRACE delivers large gains on grounding-sensitive settings.
• Score: 82.58503785830632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models increasingly succeed on multimodal reasoning benchmarks, yet their visual evidence often becomes unstable once it enters the language stack, weakening evidence-grounded reasoning. To understand this fragility, we examine the internal dynamics of VLMs through a mechanistic lens and uncover a stable three-stage redistribution of multimodal attention focus across depth: an early question-conditioned organization, a critical middle visual-dominant relay, and a late return to answer formation. We operationalize the middle phase as the Visual Relay Window (VRW), and show that its geometry varies with task demand, is causally tied to grounded generation, and distinguishes unsupported answers from stronger reasoning trajectories. Guided by this internal rhythm, we propose TRACE, a task-adaptive inference-time control framework with lightweight trained modules. It reshapes relay allocation during prefill and preserves assembled visual support after handoff during decoding. Across four open-weight VLM backbones and seven benchmarks, TRACE delivers large gains on grounding-sensitive settings, improving them by 4.33 points on average and by up to 6.6 points, while also improving reasoning-heavy tasks. These results show that explicitly controlling multimodal focus across depth offers a unified and effective mechanism for strengthening evidence-grounded multimodal reasoning.

Related papers

• Is Visual Prompting All You Need? Studying VLM Spatial Reasoning under Progressive Visual Scaffolds [12.841957617988148]
  We study how the visual presentation of a task shapes model performance and failure modes when the underlying reasoning problem is unchanged.<n>We introduce lightweight input-side scaffolds that preserve the visual modality while making spatial structure more accessible.<n>We find that visual presentation is a central factor that determines whether VLM benchmarks measure grounded perception, downstream reasoning, or a mixture of both.
  arXiv  Detail & Related papers  (2026-08-21T14:40:26Z)
• Are Reasoning Vision-Language Models Robust to Semantic Visual Distractions? [34.294060886423985]
  Reasoning Vision-Language Models (VLMs) achieve strong performance on complex multimodal tasks.<n>We introduce textbfDistract-Bench, a benchmark for evaluating VLM robustness to textbfsemantic visual distractions<n>Our results show that Distract-Bench exposes a robustness failure distinct from vision corruptions.
  arXiv  Detail & Related papers  (2026-06-08T00:40:32Z)
• Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention [41.546578522790114]
  Reinforcement learning with verifiable rewards (RLVR) has emerged as a promising paradigm for advancing complex reasoning in large language models.<n>We propose Faithful-MR1, a training framework that anchors and reinforces visual attention to address both halves of faithful multimodal reasoning.
  arXiv  Detail & Related papers  (2026-05-21T07:10:18Z)
• CAVE: A Structured Credit Assignment Approach for Fragmented Visual Evidence Reasoning [13.024838183478858]
  We propose Credit Assignment for Visual Evidence (CAVE) for interleaved visual reasoning.<n>CAVE evaluates the contribution of intermediate steps at the action level via three complementary reasoning process signals.<n>We construct TRACER-Bench, which covers four nonlocal and semantically confusable reasoning dimensions.
  arXiv  Detail & Related papers  (2026-05-13T16:50:24Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification [55.357038267439684]
  Visual Re-Examination (VRE) is a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs.<n>VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings.
  arXiv  Detail & Related papers  (2026-03-27T12:22:13Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs [55.61018839017648]
  Chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks.<n>Existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies.<n>We propose SAYO, a visual reasoning model trained with a reinforcement learning framework that introduces a region-level visual attention-based reward.
  arXiv  Detail & Related papers  (2026-02-09T03:33:23Z)
• Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning [53.790502697674754]
  We propose Take-along Visual Conditioning (TVC), a strategy that shifts image input to critical reasoning stages.<n>TVC helps the model retain attention to the visual components throughout the reasoning.<n>Our approach achieves state-of-the-art performance on average across five mathematical reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-03-17T16:45:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.