Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation

• URL: http://arxiv.org/abs/2606.08719v1
• Date: Sun, 07 Jun 2026 16:29:49 GMT
• Title: Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation
• Abstract summary: ''Thinking with Images'' has emerged as an effective paradigm for fine-grained visual reasoning.<n>We ask whether the reasoning benefits of ''Thinking with Images'' can be internalized through Thinking with Imagination.<n>We propose Imagine-OPD, an on-policy self-distillation framework in which a teacher plays the role of a ''Thinking with Images'' reasoner.
• Score: 19.81721375367558
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: ''Thinking with Images'' has emerged as an effective paradigm for fine-grained visual reasoning: by explicitly zooming into relevant regions and reasoning over crops, models can access local evidence that is difficult to recover from a single global image. However, this benefit comes with redundant tool invocations and longer inference traces. Moreover, when such behaviors are learned mainly from outcome reward, the resulting intermediate crops or visual cues can be noisy or fail to faithfully capture task-relevant visual evidence. In this work, we ask whether the reasoning benefits of ''Thinking with Images'' can be internalized through Thinking with Imagination: an internal process that decides where to look and imagines what visual cues closer inspection would reveal without actually invoking tools. We propose Imagine-OPD, an on-policy self-distillation framework in which a teacher plays the role of a ''Thinking with Images'' reasoner during training: it receives privileged zoomed evidence views derived from annotated regions, and supervises the model's own imagination reasoning trajectories. Imagine-OPD does not require an external teacher or high-quality imagination demonstrations. Experiments on vision-centric benchmarks show that Imagine-OPD achieves the best average performance among compared models while significantly reducing inference overhead compared with ''Thinking with Images'' methods.

Related papers

• Thinking with Visual Grounding [55.778136782175615]
  We introduce visually grounded thinking, a reasoning process in which models interleave natural-language thoughts with explicit point or box groundings of the visual evidence used at each step.<n>To train this behavior, we construct a scalable synthesis pipeline that distills correct visual reasoning traces.<n>We propose grounding-aware reinforcement learning, which combines answer correctness rewards with dense grounding rewards that score whether generated object references match the correct image evidence.
  arXiv  Detail & Related papers  (2026-06-15T02:28:34Z)
• How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning [18.15809161363394]
  Cross-view spatial reasoning remains a weak spot for vision-language models.<n>We ask how to make visual thinking matter, and what kind of visual thinking works best.
  arXiv  Detail & Related papers  (2026-05-26T17:20:05Z)
• Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation [71.92541392470103]
  Multimodal Large Language Models (MLLMs) still struggle with fine-grained visual understanding.<n>We propose Vision-OPD (Vision On-Policy Distillation), a regional-to-global self-distillation framework.<n>Vision-OPD instantiates two conditional policies from the same MLLM: a crop-conditioned teacher and a full-image-conditioned student.
  arXiv  Detail & Related papers  (2026-05-18T17:57:04Z)
• Improving Visual Reasoning with Iterative Evidence Refinement [41.18410988040976]
  Vision language models (VLMs) are increasingly capable of reasoning over images.<n>But robust visual reasoning often requires re-grounding intermediate steps in the underlying visual evidence.<n>We propose an end-to-end self-revisit framework, SIEVE, that trains models to re-engage image evidence through internal representations.
  arXiv  Detail & Related papers  (2026-03-14T21:08:21Z)
• When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoning [108.73849507002195]
  We present an in-depth analysis of test-time visual imagination as a controllable resource for spatial reasoning.<n>We study when static visual evidence is sufficient, when imagination improves reasoning, and how excessive or unnecessary imagination affects accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-02-09T03:21:48Z)
• Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images [53.373427633330515]
  We propose DRIM, a model that enables deep but reliable multi-turn reasoning when thinking with images in its multimodal CoT.<n>Based on a high-resolution image dataset, we construct high-difficulty and verifiable visual question-answer pairs.<n>In the SFT stage, we collect tool trajectories as cold-start data, guiding a multi-turn reasoning pattern.<n>In the RL stage, we introduce redundancy-penalized policy optimization, which incentivizes the model to develop a self-reflective reasoning pattern.
  arXiv  Detail & Related papers  (2025-12-19T07:44:43Z)
• From Illusion to Intention: Visual Rationale Learning for Vision-Language Reasoning [19.84653798433995]
  We propose Visual Rationale Learning (ViRL), an end-to-end paradigm that grounds training in the visual rationale itself.<n>ViRL integrates (1) Process Supervision with ground-truth rationales, (2) Objective Alignment via step-level reward shaping, and (3) Fine-Grained Credit Assignment to distinguish correct, redundant, and erroneous actions.<n>This work establishes visual rationalization as a task-agnostic, process-grounded paradigm for building transparent, verifiable, and trustworthy vision-language models.
  arXiv  Detail & Related papers  (2025-11-28T09:52:56Z)
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning [72.81576836419373]
  Chain-of-Thought (CoT) reasoning can be used to link visual cues across multiple images.<n>We adapt rule-based reinforcement learning for Vision-Language Models (VLMs)<n>Our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.
  arXiv  Detail & Related papers  (2025-06-27T17:59:27Z)
• DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning [11.242852367476015]
  DeepEyes is a model with "thinking with images" capabilities incentivized through end-to-end reinforcement learning.<n>We propose a tool-use-oriented data selection mechanism and a reward strategy to encourage successful tool-assisted reasoning trajectories.<n>DeepEyes achieves significant performance gains on fine-grained perception and reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-05-20T13:48:11Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.