Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

• URL: http://arxiv.org/abs/2608.15869v1
• Date: Sun, 16 Aug 2026 17:35:38 GMT
• Title: Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning
• Abstract summary: Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments.<n>We ask whether models can learn to think visually during training while reasoning directly at inference.<n>We introduce Internalized Visual Thinking (IVT), a framework that jointly optimize textual prediction and next-embedding prediction over unlabeled videos.
• Score: 15.521458272682308
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal large language models increasingly use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments. By generating intermediate reasoning images, Visual CoT provides an intuitive mechanism for visual foresight but introduces substantial inference overhead, which is particularly problematic for proactive video reasoning. We ask whether models can learn to think visually during training while reasoning directly at inference. We introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos. Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent. At inference, IVT generates the answer directly without synthesizing or re-encoding future frames. We conduct controlled studies across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives. IVT improves over direct-answer fine-tuning on all six evaluation settings while retaining the same inference pathway. Compared with explicit Visual CoT, IVT achieves comparable or better performance and reduces average end-to-end latency by more than 5x. Together, our findings suggest that explicit pixel-space generation at inference time, as used in visual chain-of-thought, may not be necessary for effective proactive video reasoning. Predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.

Related papers

• From Frames to Temporal Graphs: In-Context Egocentric Action Recognition with Vision-Language Models [67.00247936789127]
  Action reasoning in egocentric video requires capturing fine-grained transitions of hand-object interactions.<n>We propose to decouple visual perception from symbolic reasoning by converting videos into Temporal Action Graphs.
  arXiv  Detail & Related papers  (2026-06-13T18:00:42Z)
• VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoning [10.735514435202662]
  Video reasoning aims to understand complex temporal events and causal relationships within videos.<n>Existing Chain-of-Thought (CoT) video reasoning methods rely on text-only information for logical deduction.<n>We propose Visual-Textual Interleaved CoT framework to integrate textual reasoning steps with corresponding visual frames.
  arXiv  Detail & Related papers  (2026-06-04T05:55:15Z)
• STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models [81.32710031596591]
  Video reasoning tasks require tracking motion, temporal order, and evolving visual states across frames.<n>Existing methods built on vision-language models (LVLMs) often address this challenge by externalizing reasoning through chain-of-thought (CoT)<n>We propose STORMS, a two-stage framework that teaches LVLMs to reason through bounded continuous latent trajectories instead of explicit textual CoT.
  arXiv  Detail & Related papers  (2026-05-25T16:33:00Z)
• Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs [60.93949629734977]
  We propose Visual Contrastive Self-Taught Reasoner (VC-STaR) to mitigate hallucinations in model-generated rationales.<n>We collect a diverse suite of VQA datasets, curate contrastive pairs according to multi-modal similarity, and generate rationales using VC-STaR.<n>Extensive experiments show that VC-STaR not only outperforms existing self-improving approaches but also surpasses models finetuned on the SoTA visual reasoning datasets.
  arXiv  Detail & Related papers  (2026-03-03T03:18:31Z)
• TwiFF (Think With Future Frames): A Large-Scale Dataset for Dynamic Visual Reasoning [39.81570843186615]
  Visual Chain-of-Thought (VCoT) has emerged as a promising paradigm for enhancing multimodal reasoning by integrating visual perception into intermediate reasoning steps.<n>We propose TwiFF, the first large-scale, temporally grounded VCoT dataset derived from $2.7$ million video clips.<n>We show that TwiFF significantly outperforms existing VCoT methods and Textual Chain-of-Thought baselines on dynamic reasoning tasks.
  arXiv  Detail & Related papers  (2026-02-11T09:20:04Z)
• Interleaved Latent Visual Reasoning with Selective Perceptual Modeling [42.93438443502933]
  Interleaved reasoning paradigms enhance Multimodal Large Language Models (MLLMs) with visual feedback but are hindered by the prohibitive computational cost.<n>A promising alternative, latent visual reasoning, circumvents this bottleneck yet currently forces a critical trade-off.<n>We introduce Interleaved Latent Visual Reasoning (ILVR), a framework that unifies dynamic state evolution with precise perceptual modeling.
  arXiv  Detail & Related papers  (2025-12-05T12:09:39Z)
• FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving [19.81442567260658]
  We propose a visual-temporalT framework that enables VLAs to think in images.<n>On nuScenes and NAVSIM, FSDrive improves accuracy and reduces collisions.
  arXiv  Detail & Related papers  (2025-05-23T09:55:32Z)
• Predicting Long-horizon Futures by Conditioning on Geometry and Time [49.86180975196375]
  We explore the task of generating future sensor observations conditioned on the past. We leverage the large-scale pretraining of image diffusion models which can handle multi-modality. We create a benchmark for video prediction on a diverse set of videos spanning indoor and outdoor scenes.
  arXiv  Detail & Related papers  (2024-04-17T16:56:31Z)
• Unleashing Text-to-Image Diffusion Models for Visual Perception [84.41514649568094]
  VPD (Visual Perception with a pre-trained diffusion model) is a new framework that exploits the semantic information of a pre-trained text-to-image diffusion model in visual perception tasks. We show that VPD can be faster adapted to downstream visual perception tasks using the proposed VPD.
  arXiv  Detail & Related papers  (2023-03-03T18:59:47Z)
• See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning [60.43585179885355]
  We propose a novel framework named Interactive Prompting Visual Reasoner (IPVR) for few-shot knowledge-based visual reasoning. IPVR contains three stages, see, think and confirm. We conduct experiments on a range of knowledge-based visual reasoning datasets.
  arXiv  Detail & Related papers  (2023-01-12T18:59:50Z)
• Unified Recurrence Modeling for Video Action Anticipation [16.240254363118016]
  We propose a unified recurrence modeling for video action anticipation via message passing framework. Our proposed method outperforms previous works on the large-scale EPIC-Kitchen dataset.
  arXiv  Detail & Related papers  (2022-06-02T12:16:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.