Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generation

• URL: http://arxiv.org/abs/2512.03040v2
• Date: Thu, 11 Dec 2025 08:18:10 GMT
• Title: Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generation
• Abstract summary: Video generative models conditioned solely on video-based scene context can perform complex spatial tasks.<n>We present Video4Spatial, a framework showing that video diffusion models conditioned solely on video-based scene context can perform complex spatial tasks.
• Score: 32.94087966263353
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We investigate whether video generative models can exhibit visuospatial intelligence, a capability central to human cognition, using only visual data. To this end, we present Video4Spatial, a framework showing that video diffusion models conditioned solely on video-based scene context can perform complex spatial tasks. We validate on two tasks: scene navigation - following camera-pose instructions while remaining consistent with 3D geometry of the scene, and object grounding - which requires semantic localization, instruction following, and planning. Both tasks use video-only inputs, without auxiliary modalities such as depth or poses. With simple yet effective design choices in the framework and data curation, Video4Spatial demonstrates strong spatial understanding from video context: it plans navigation and grounds target objects end-to-end, follows camera-pose instructions while maintaining spatial consistency, and generalizes to long contexts and out-of-domain environments. Taken together, these results advance video generative models toward general visuospatial reasoning.

Related papers

• Towards Physical Understanding in Video Generation: A 3D Point Regularization Approach [54.559847511280545]
  We present a novel video generation framework that integrates 3-dimensional geometry and dynamic awareness.<n>To achieve this, we augment 2D videos with 3D point trajectories and align them in pixel space.<n>The resulting 3D-aware video dataset, PointVid, is then used to fine-tune a latent diffusion model.
  arXiv  Detail & Related papers  (2025-02-05T21:49:06Z)
• VideoOrion: Tokenizing Object Dynamics in Videos [34.96534298857146]
  We present VideoOrion, a Video Large Language Model (Video-LLM) that explicitly captures the key semantic information in videos.<n>VideoOrion employs expert vision models to extract object dynamics through a detect-segment-track pipeline.<n>Our method addresses the persistent challenge in Video-LLMs of efficiently compressing high-dimensional video data into semantic tokens.
  arXiv  Detail & Related papers  (2024-11-25T07:32:02Z)
• Dense Video Object Captioning from Disjoint Supervision [77.47084982558101]
  We propose a new task and model for dense video object captioning. This task unifies spatial and temporal localization in video. We show how our model improves upon a number of strong baselines for this new task.
  arXiv  Detail & Related papers  (2023-06-20T17:57:23Z)
• SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos [23.64091569954785]
  We introduce SAVi++, an object-centric video model which is trained to predict depth signals from a slot-based video representation. By using sparse depth signals obtained from LiDAR, SAVi++ is able to learn emergent object segmentation and tracking from videos in the real-world Open dataset.
  arXiv  Detail & Related papers  (2022-06-15T18:57:07Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• Video Autoencoder: self-supervised disentanglement of static 3D structure and motion [60.58836145375273]
  A video autoencoder is proposed for learning disentan- gled representations of 3D structure and camera pose from videos. The representation can be applied to a range of tasks, including novel view synthesis, camera pose estimation, and video generation by motion following.
  arXiv  Detail & Related papers  (2021-10-06T17:57:42Z)
• Video Exploration via Video-Specific Autoencoders [60.256055890647595]
  We present video-specific autoencoders that enables human-controllable video exploration. We observe that a simple autoencoder trained on multiple frames of a specific video enables one to perform a large variety of video processing and editing tasks.
  arXiv  Detail & Related papers  (2021-03-31T17:56:13Z)
• Visual Relation Grounding in Videos [86.06874453626347]
  We explore a novel named visual Relation Grounding in Videos (RGV) This task aims at providing supportive visual facts for other video-language tasks (e.g., video grounding and video question answering) We tackle challenges by collaboratively optimizing two sequences of regions over a constructed hierarchical-temporal region. Experimental results demonstrate our model can not only outperform baseline approaches significantly, but also produces visually meaningful facts.
  arXiv  Detail & Related papers  (2020-07-17T08:20:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.