Iterative Visual Thinking: Teaching Vision-Language Models Spatial Self-Correction through Visual Feedback

• URL: http://arxiv.org/abs/2606.13156v1
• Date: Thu, 11 Jun 2026 10:27:29 GMT
• Title: Iterative Visual Thinking: Teaching Vision-Language Models Spatial Self-Correction through Visual Feedback
• Abstract summary: Vision-language models (VLMs) achieve strong singleshot spatial grounding, yet lack any mechanism to observe and correct their own predictions.<n>We propose Iterative Visual Thinking (IVT), a closed-loop framework in which the model predicts a bounding box, observes the prediction rendered on the image, and iteratively refines through visual feedback.<n>All training uses only 2,400 samples on a single GPU, demonstrating that spatial self-correction is a learnable capability that can be instilled at modest scale.
• Score: 0.023227405857540805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language models (VLMs) achieve strong singleshot spatial grounding, yet lack any mechanism to observe and correct their own predictions. We find that naively prompting a VLM to iterate over rendered visualizations of its predictions causes catastrophic failure: Acc@0.5 on referring expression comprehension collapses from 79.6% to 48.7% (a 31 percentage point drop), revealing a fundamental gap between grounding capability and self-correction ability. We propose Iterative Visual Thinking (IVT), a closed-loop framework in which the model predicts a bounding box, observes the prediction rendered on the image, and iteratively refines through visual feedback. A two-phase training recipe closes the self-correction gap: first, we exploit the base model's own predictions as realistic errors and prompt a teacher VLM to generate corrective reasoning traces, yielding supervised data without human annotation; second, we apply Group Relative Policy Optimization (GRPO) with a simple IoU reward to stabilize multi-step refinement. On a mixed benchmark spanning RefCOCOg, Ref-Adv, and Ref-L4 (505 test samples), SFT warm-up with IVT surpasses the single-shot base model on every metric: Acc@0.5 rises to 82.0% (+2.4pp), Acc@0.7 to 74.1% (+3.2pp), and Acc@0.9 to 48.3% (+2.8pp). GRPO further reduces per-step IoU degradation by 5x, stabilizing the refinement trajectory. All training uses only 2,400 samples on a single GPU, demonstrating that spatial self-correction is a learnable capability that can be instilled at modest scale.

Related papers

• Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning [48.438568700670835]
  The standard way to read latent knowledge out of a model can systematically overstate what a vision-language model grounds in the image.<n>We show this on spatial reasoning, where the error is invisible to both probing and steering yet exposed by a one-line causal control.
  arXiv  Detail & Related papers  (2026-06-30T07:33:18Z)
• UniviewVLA: A Unified Multiview Vision-Language-Action Model with World Modeling [33.6086006552446]
  We propose UniviewVLA, a unified multiview Vision-Language-Action model with world modeling.<n>By leveraging generated multiview future views, UniviewVLA reveals occluded cues and models future scene evolution.<n>UniviewVLA achieves 95.8% on LIBERO and 4.60 on CALVIN ABCD to D, both standard occlusion-free benchmarks.
  arXiv  Detail & Related papers  (2026-06-19T14:53:40Z)
• Bridging the Generalization Gap in Adverse Weather Segmentation: A Training Recipe Perspective [9.751481477523956]
  This paper describes our approach for the 8th UG2+ Workshop (CVPR 2026) Track2, which targets semantic segmentation of outdoor scenes degraded by five weather conditions.<n>A central challenge we observe is a severe generalization gap -- models that perform well on the validation set often collapse on the test set.<n>We investigate whether a carefully designed training recipe, rather than architectural complexity, can address this gap.
  arXiv  Detail & Related papers  (2026-05-27T04:55:34Z)
• CrossVLA: Cross-Paradigm Post-Training and Inference Optimization for Vision-Language-Action Models [7.489020109808801]
  We present CrossVLA, an empirical study of cross-paradigm Vision-Language-Action (VLA) post-training.<n>Three contributions: (i) a surrogate flow-matching log-probability estimator that lets DPO operate on continuous-action backbones without probability-flow ODE integration; (ii) a head-to-head comparison of LoRA and DoRA as the parameter-efficient layer for VLA DPO; and (iii) an inference-time anatomy showing the denoise loop dominates 78.6% of sample_actions latency and prefix-K/V caching a la
  arXiv  Detail & Related papers  (2026-05-21T01:02:41Z)
• One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness [12.183451602438753]
  We show that simple lexical constraints (banning a single punctuation character or common word) cause instruction-tuned LLMs to collapse their responses.<n>We show that base models show no systematic collapse under identical constraints, with effects that are small, noisy, and bidirectional.
  arXiv  Detail & Related papers  (2026-04-14T17:40:01Z)
• Understanding Degradation with Vision Language Model [56.09241449206817]
  Understanding visual degradations is a critical yet challenging problem in computer vision.<n>We introduce DU-VLM, a multimodal chain-of-thought model trained with supervised fine-tuning and reinforcement learning.<n>We also introduce textbfDU-110k, a large-scale dataset comprising 110,000 clean-degraded pairs with grounded physical annotations.
  arXiv  Detail & Related papers  (2026-02-04T13:51:15Z)
• VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling [60.341503853471494]
  We show that vision-language-action models degrade sharply under novel camera viewpoints and visual perturbations.<n>We propose a one-shot adaptation framework that recalibrates visual representations through lightweight, learnable updates.
  arXiv  Detail & Related papers  (2025-12-02T16:16:13Z)
• Token-Level Inference-Time Alignment for Vision-Language Models [58.41370989069588]
  Vision-Language Models (VLMs) have become essential backbones of modern multimodal intelligence.<n>We present TITA, a lightweight framework that freezes the base VLM and instead trains a reward model to approximate its distribution.<n>During inference, implicit preference signals are extracted as log-probability ratios between the reward model and the target VLM, yielding dense autoregressive feedback.
  arXiv  Detail & Related papers  (2025-10-20T09:58:03Z)
• Sherlock: Self-Correcting Reasoning in Vision-Language Models [27.122890248991556]
  Reasoning Vision-Language Models (VLMs) have shown promising performance on complex multimodal tasks.<n>They are highly sensitive to reasoning errors, require large volumes of annotated data or accurate verifiers, and struggle to generalize.<n>We introduce Sherlock, a self-correction and self-improvement training framework.<n>Built on the Llama3.2-Vision-11B model, Sherlock achieves remarkable results across eight benchmarks.
  arXiv  Detail & Related papers  (2025-05-28T17:58:03Z)
• Self-Play Preference Optimization for Language Model Alignment [75.83359213697854]
  Recent advancements suggest that directly working with preference probabilities can yield a more accurate reflection of human preferences. We propose a self-play-based method for language model alignment, which treats the problem as a constant-sum two-player game. Our approach, dubbed Self-Play Preference Optimization (SPPO), utilizes iterative policy updates to provably approximate the Nash equilibrium.
  arXiv  Detail & Related papers  (2024-05-01T17:59:20Z)
• Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations [58.442103936918805]
  We show that Attention Mask Consistency produces superior visual grounding results than previous methods. AMC is effective, easy to implement, and is general as it can be adopted by any vision-language model.
  arXiv  Detail & Related papers  (2022-06-30T17:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.