Spatial Causal Prediction in Video

• URL: http://arxiv.org/abs/2603.03944v1
• Date: Wed, 04 Mar 2026 11:09:39 GMT
• Title: Spatial Causal Prediction in Video
• Authors: Yanguang Zhao, Jie Yang, Shengqiong Wu, Shutong Hu, Hongbo Qiu, Yu Wang, Guijia Zhang, Tan Kai Ze, Hao Fei, Chia-Wen Lin, Mong-Li Lee, Wynne Hsu,
• Abstract summary: We introduce a new task paradigm that challenges models to reason beyond observation and predict spatial causal outcomes.<n>We construct a benchmark comprising 2,500 QA pairs across 1,181 videos spanning diverse viewpoints, scenes, and causal directions.<n>Through comprehensive experiments on 23 state-of-the-art models, we reveal substantial gaps between human and model performance limited temporal extrapolation, and weak causal grounding.
• Score: 56.22332198384257
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Spatial reasoning, the ability to understand spatial relations, causality, and dynamic evolution, is central to human intelligence and essential for real-world applications such as autonomous driving and robotics. Existing studies, however, primarily assess models on visible spatio-temporal understanding, overlooking their ability to infer unseen past or future spatial states. In this work, we introduce Spatial Causal Prediction (SCP), a new task paradigm that challenges models to reason beyond observation and predict spatial causal outcomes. We further construct SCP-Bench, a benchmark comprising 2,500 QA pairs across 1,181 videos spanning diverse viewpoints, scenes, and causal directions, to support systematic evaluation. Through comprehensive experiments on {23} state-of-the-art models, we reveal substantial gaps between human and model performance, limited temporal extrapolation, and weak causal grounding. We further analyze key factors influencing performance and propose perception-enhancement and reasoning-guided strategies toward advancing spatial causal intelligence. The project page is https://guangstrip.github.io/SCP-Bench.

Related papers

• Imagine in Space: Exploring the Frontier of Spatial Intelligence and Reasoning Efficiency in Vision Language Models [23.12717700882611]
  spatial reasoning is a fundamental component of human cognition.<n>Current large language models (LLMs) and vision language models (VLMs) have demonstrated remarkable reasoning capabilities across logical inference, problem solving, and decision making.<n>We hypothesize that imagination, the internal simulation of spatial states, is the dominant reasoning mechanism within a spatial world model.
  arXiv  Detail & Related papers  (2025-11-16T03:09:55Z)
• Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks [108.15756345836901]
  We provide a comprehensive review of multimodal spatial reasoning tasks with large models.<n>We review advances in embodied AI, including vision-language navigation and action models.<n>We consider emerging modalities such as audio and egocentric video, which contribute to novel spatial understanding through new sensors.
  arXiv  Detail & Related papers  (2025-10-29T17:55:43Z)
• How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective [103.44502230776352]
  We present a systematic investigation of Visual Spatial Reasoning (VSR) in Vision-Language Models (VLMs)<n>We categorize spatial intelligence into three levels of capability, ie, basic perception, spatial understanding, spatial planning, and curate SIBench, a spatial intelligence benchmark encompassing nearly 20 open-source datasets across 23 task settings.
  arXiv  Detail & Related papers  (2025-09-23T12:00:14Z)
• CausalVQA: A Physically Grounded Causal Reasoning Benchmark for Video Models [4.889577550694335]
  CausalVQA is a benchmark dataset for video question answering (VQA)<n>It consists of question-answer pairs that probe models' understanding of causality in the physical world.
  arXiv  Detail & Related papers  (2025-06-11T17:10:36Z)
• OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models [17.976302783133956]
  We introduce OmniSpatial, a benchmark for spatial reasoning grounded in cognitive psychology.<n>It covers four major categories: dynamic reasoning, complex spatial logic, spatial interaction, and perspective-taking.<n>Through careful manual annotation, we construct over 8.4K question-answer pairs.
  arXiv  Detail & Related papers  (2025-06-03T17:58:29Z)
• SITE: towards Spatial Intelligence Thorough Evaluation [121.1493852562597]
  Spatial intelligence (SI) represents a cognitive ability encompassing the visualization, manipulation, and reasoning about spatial relationships.<n>We introduce SITE, a benchmark dataset towards SI Thorough Evaluation.<n>Our approach to curating the benchmark combines a bottom-up survey about 31 existing datasets and a top-down strategy drawing upon three classification systems in cognitive science.
  arXiv  Detail & Related papers  (2025-05-08T17:45:44Z)
• Where Am I and What Will I See: An Auto-Regressive Model for Spatial Localization and View Prediction [60.964512894143475]
  We present Generative Spatial Transformer ( GST), a novel auto-regressive framework that jointly addresses spatial localization and view prediction. Our model simultaneously estimates the camera pose from a single image and predicts the view from a new camera pose, effectively bridging the gap between spatial awareness and visual prediction.
  arXiv  Detail & Related papers  (2024-10-24T17:58:05Z)
• Causal Reasoning Meets Visual Representation Learning: A Prospective Study [117.08431221482638]
  Lack of interpretability, robustness, and out-of-distribution generalization are becoming the challenges of the existing visual models. Inspired by the strong inference ability of human-level agents, recent years have witnessed great effort in developing causal reasoning paradigms. This paper aims to provide a comprehensive overview of this emerging field, attract attention, encourage discussions, bring to the forefront the urgency of developing novel causal reasoning methods.
  arXiv  Detail & Related papers  (2022-04-26T02:22:28Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Development of Human Motion Prediction Strategy using Inception Residual Block [1.0705399532413613]
  We propose an Inception Residual Block (IRB) to detect temporal features in human poses. Our main contribution is to propose a residual connection between input and the output of the inception block to have a continuity between the previously observed pose and the next predicted pose. With this proposed architecture, it learns prior knowledge much better about human poses and we achieve much higher prediction accuracy as detailed in the paper.
  arXiv  Detail & Related papers  (2021-08-09T12:49:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.