How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

• URL: http://arxiv.org/abs/2605.27310v1
• Date: Tue, 26 May 2026 17:20:05 GMT
• Title: How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning
• Abstract summary: Cross-view spatial reasoning remains a weak spot for vision-language models.<n>We ask how to make visual thinking matter, and what kind of visual thinking works best.
• Score: 18.15809161363394
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry needed for the task. Thinking with images aims to address this by generating an intermediate thinking image, but recent work shows that models often ignore the visual evidence in these traces. We therefore ask how to make visual thinking matter, and what kind of visual thinking works best. We study these questions in unified multimodal models (UMMs), which natively support interleaved image-text generation. For the first question, we propose View Dropout (VDrop), a training-time intervention that hides parts of one input view from the answer span while keeping them visible to the thinking-image tokens. This encourages the model to use the thinking image when answering, instead of relying only on the input views. Once the thinking image is used for answer prediction, we study which type of visual thinking is most effective. We frame this as a learnability-informativeness tradeoff and compare three thinking-image variants: top-down, panoramic, and point-matching renderings. Trained on synthetic scenes and evaluated on five real-world out-of-domain benchmarks, panoramic visual thinking with VDrop is the only configuration that is both informative and learnable, and it achieves the best out-of-domain generalization.

Related papers

• Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models [97.0559653915626]
  We introduce three new vision-centric evaluation benchmarks in temporal frame retrieval, video future prediction, and causal memory distortion.<n>We show that current frontier models exhibit significant weakness when attempting to reason through visual queries, rather than text.
  arXiv  Detail & Related papers  (2026-09-15T12:41:08Z)
• Thinking with Visual Grounding [55.778136782175615]
  We introduce visually grounded thinking, a reasoning process in which models interleave natural-language thoughts with explicit point or box groundings of the visual evidence used at each step.<n>To train this behavior, we construct a scalable synthesis pipeline that distills correct visual reasoning traces.<n>We propose grounding-aware reinforcement learning, which combines answer correctness rewards with dense grounding rewards that score whether generated object references match the correct image evidence.
  arXiv  Detail & Related papers  (2026-06-15T02:28:34Z)
• Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination [39.713386491199884]
  Vision-Language Models (VLMs) often produce self-reflective statements like "let me check the figure again" during reasoning.<n>Do such statements trigger genuine visual re-examination, or are they merely learned textual patterns?<n>We investigate this via VisualSwap, an image-swap probing framework.
  arXiv  Detail & Related papers  (2026-05-15T11:31:14Z)
• VisualScratchpad: Inference-time Visual Concepts Analysis in Vision Language Models [81.7389759509619]
  We introduce VisualScratchpad, an interactive interface for visual concept analysis during inference.<n>We apply sparse autoencoders to the vision encoder and link the resulting visual concepts to text tokens via text-to-image attention.<n>VisualScratchpad also provides a token-latent heatmap view that suggests a sufficient set of latents for effective concept ablation in causal analysis.
  arXiv  Detail & Related papers  (2026-03-07T20:52:09Z)
• MentisOculi: Revealing the Limits of Reasoning with Mental Imagery [63.285794947638614]
  We develop MentisOculi, a suite of multi-step reasoning problems amenable to visual solution.<n> evaluating visual strategies ranging from latent tokens to explicit generated imagery, we find they generally fail to improve performance.<n>Our findings suggest that despite their inherent appeal, visual thoughts do not yet benefit model reasoning.
  arXiv  Detail & Related papers  (2026-02-02T18:49:06Z)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought [118.71264263478083]
  We propose MIRA, a new benchmark designed to evaluate models in scenarios where generating intermediate visual images is essential for successful reasoning.<n>We include 546 multimodal problems, annotated with intermediate visual images and final answers.
  arXiv  Detail & Related papers  (2025-11-04T18:00:51Z)
• BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception [67.89135437537179]
  We introduce BLINK-Twice, a vision-centric reasoning benchmark grounded in challenging perceptual tasks.<n>Instead of relying on external knowledge, our tasks require models to reason from visual content alone.<n>Compared to prior perception benchmarks, it moves beyond shallow perception and requires fine-grained observation and analytical reasoning.
  arXiv  Detail & Related papers  (2025-10-10T13:14:13Z)
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning [72.81576836419373]
  Chain-of-Thought (CoT) reasoning can be used to link visual cues across multiple images.<n>We adapt rule-based reinforcement learning for Vision-Language Models (VLMs)<n>Our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.
  arXiv  Detail & Related papers  (2025-06-27T17:59:27Z)
• Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens [44.19323180593379]
  Vision-language models (VLMs) excel at multimodal understanding, yet their text-only decoding forces them to verbalize visual reasoning.<n>Recent attempts train VLMs to render explicit images, but the heavy image-generation pre-training often hinders the reasoning ability.<n>We present a Machine Mental Imagery framework, dubbed as Mirage, which augments VLM decoding with latent visual tokens alongside ordinary text.
  arXiv  Detail & Related papers  (2025-06-20T17:59:31Z)
• v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning [27.688428439248607]
  We introduce v1, a lightweight extension that enables active visual referencing through a simple point-and-copy approach.<n>This allows the model to identify relevant image patches and copy their embeddings back into the reasoning stream.<n>Our pointing strategy lets the MLLM directly select image patches using their semantic representations as keys, keeping perceptual evidence embedded in the same space as the model's reasoning.
  arXiv  Detail & Related papers  (2025-05-24T19:30:47Z)
• Localization vs. Semantics: Visual Representations in Unimodal and Multimodal Models [57.08925810659545]
  We conduct a comparative analysis of the visual representations in existing vision-and-language models and vision-only models. Our empirical observations suggest that vision-and-language models are better at label prediction tasks. We hope our study sheds light on the role of language in visual learning, and serves as an empirical guide for various pretrained models.
  arXiv  Detail & Related papers  (2022-12-01T05:00:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.