Improving Visual Reasoning with Iterative Evidence Refinement

• URL: http://arxiv.org/abs/2603.14117v1
• Date: Sat, 14 Mar 2026 21:08:21 GMT
• Title: Improving Visual Reasoning with Iterative Evidence Refinement
• Abstract summary: Vision language models (VLMs) are increasingly capable of reasoning over images.<n>But robust visual reasoning often requires re-grounding intermediate steps in the underlying visual evidence.<n>We propose an end-to-end self-revisit framework, SIEVE, that trains models to re-engage image evidence through internal representations.
• Score: 41.18410988040976
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision language models (VLMs) are increasingly capable of reasoning over images, but robust visual reasoning often requires re-grounding intermediate steps in the underlying visual evidence. Recent approaches typically rely on external image operations such as zooming or cropping to re-access fine-grained details during inference, which requires additional image re-encoding and can disrupt the reasoning trajectory. We argue that VLMs already provide strong internal signals for identifying and reusing visual evidence, and that these signals can be directly leveraged to support image-grounded reasoning. Motivated by this insight, we propose an end-to-end self-revisit framework, SIEVE, that trains models to re-engage image evidence through internal representations. SIEVE automatically extracts embeddings of salient image regions and injects them into the reasoning chain when additional grounding is needed, enabling later steps to condition on relevant visual cues without external tool calls or re-encoding. We use reinforcement learning to teach the model when to trigger visual revisiting and which region embeddings to retrieve and insert during the reasoning process. Experiments on multiple visual reasoning benchmarks, together with perception, reasoning, and hallucination evaluations, show that SIEVE yields consistent gains, improving performance by 8 percent on average across several benchmarks.

Related papers

• GLaQ: Grounding Latent Queries in Visual Evidence for Multimodal Reasoning [27.232897918541067]
  Chain-of-thought reasoning has substantially improved the problem-solving capabilities of multimodal large language models.<n>Fine-grained visual evidence, however, remains difficult to preserve and reuse across text-based reasoning steps.<n>We introduce GLaQ, a grounded latent-query framework that replaces sequential latent rollout with a fixed set of context-conditioned queries.
  arXiv  Detail & Related papers  (2026-08-16T04:01:08Z)
• Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning [50.56518447675037]
  Multimodal large language models (MLLMs) increasingly rely on long chain-of-thought reasoning for complex tasks.<n>As reasoning sequences lengthen, models may gradually rely less on visual evidence and more on accumulated textual context, leading to visual forgetting.<n>We propose Remember-R1, a reinforcement learning framework that mitigates long-context visual forgetting by applying process-level supervision directly on the original reasoning trajectory.
  arXiv  Detail & Related papers  (2026-08-02T15:31:31Z)
• Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning [43.37915637521463]
  We propose a training-free adaptive visual gating method for unified multimodal reasoning.<n>AdaViG dynamically evaluates each triggered visual step at an early visual generation stage and aborts it when both signals are weak.
  arXiv  Detail & Related papers  (2026-07-10T22:03:01Z)
• See First, Answer Later: Visual Evidence Pre-Alignment via Sufficiency-Driven RL [19.682815357013453]
  Multimodal large language models (MLLMs) integrate strong text reasoning with visual inputs, yet their responses can be inconsistent with the underlying images.<n>We introduce Visual Evidence Pre-Alignment (VEPA), an intermediate stage between pretraining and post-training to optimize question-conditioned visual evidence descriptions.
  arXiv  Detail & Related papers  (2026-06-16T08:45:24Z)
• STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models [81.32710031596591]
  Video reasoning tasks require tracking motion, temporal order, and evolving visual states across frames.<n>Existing methods built on vision-language models (LVLMs) often address this challenge by externalizing reasoning through chain-of-thought (CoT)<n>We propose STORMS, a two-stage framework that teaches LVLMs to reason through bounded continuous latent trajectories instead of explicit textual CoT.
  arXiv  Detail & Related papers  (2026-05-25T16:33:00Z)
• Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification [55.357038267439684]
  Visual Re-Examination (VRE) is a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs.<n>VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings.
  arXiv  Detail & Related papers  (2026-03-27T12:22:13Z)
• MIRROR: Multimodal Iterative Reasoning via Reflection on Visual Regions [42.03378622674476]
  We propose the MIRROR framework for Multimodal Iterative Reasoning via Reflection On visual Regions.<n>By embedding visual reflection as a core mechanism, MIRROR is formulated as a closed-loop process comprising draft, critique, region-based verification, and revision.<n>Experiments on both general vision-language benchmarks and representative vision-language reasoning benchmarks show that MIRROR improves correctness and reduces visual hallucinations.
  arXiv  Detail & Related papers  (2026-02-21T07:56:59Z)
• VisRAG 2.0: Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation [64.82775032985485]
  Visual retrieval-augmented generation (VRAG) augments vision-language models (VLMs) with external visual knowledge to ground reasoning and reduce hallucinations.<n>Yet current VRAG systems often fail to reliably perceive and integrate evidence across multiple images, leading to weak grounding and erroneous conclusions.<n>We propose EVisRAG, an end-to-end framework that learns to reason with evidence-guided multi-image to address this issue.
  arXiv  Detail & Related papers  (2025-10-10T13:34:23Z)
• Self-Rewarding Vision-Language Model via Reasoning Decomposition [49.784411666601905]
  Vision-Language Models (VLMs) often suffer from visual hallucinations, saying things that are not actually in the image, and language shortcuts.<n>We introduce Vision-SR1, a self-rewarding method that improves visual reasoning without relying on external visual supervisions.<n>Our experiments demonstrate that Vision-SR1 improves visual reasoning, mitigates visual hallucinations, and reduces reliance on language shortcuts.
  arXiv  Detail & Related papers  (2025-08-27T08:01:03Z)
• Look-Back: Implicit Visual Re-focusing in MLLM Reasoning [15.478700750705643]
  Multimodal Large Language Models (MLLMs) have achieved remarkable progress in multimodal reasoning.<n>Current methods typically address this by explicitly injecting visual information to guide the reasoning process.<n>We introduce Look-Back, an implicit approach designed to guide MLLMs to look back" at visual information in a self-directed manner during reasoning.
  arXiv  Detail & Related papers  (2025-07-02T14:59:35Z)
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning [72.81576836419373]
  Chain-of-Thought (CoT) reasoning can be used to link visual cues across multiple images.<n>We adapt rule-based reinforcement learning for Vision-Language Models (VLMs)<n>Our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.
  arXiv  Detail & Related papers  (2025-06-27T17:59:27Z)
• Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning [53.790502697674754]
  We propose Take-along Visual Conditioning (TVC), a strategy that shifts image input to critical reasoning stages.<n>TVC helps the model retain attention to the visual components throughout the reasoning.<n>Our approach achieves state-of-the-art performance on average across five mathematical reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-03-17T16:45:12Z)
• Retrieval-Based Interleaved Visual Chain-of-Thought in Real-World Driving Scenarios [69.00444996464662]
  We propose RIV-CoT, a Retrieval-Based Interleaved Visual Chain-of-Thought method that enables vision-language models to reason using visual crops corresponding to relevant entities.<n>Our experiments demonstrate that RIV-CoT improves answer accuracy by 3.1% and reasoning accuracy by 4.6% over vanilla CoT prompting.
  arXiv  Detail & Related papers  (2025-01-08T18:31:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.