CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering

• URL: http://arxiv.org/abs/2605.23216v1
• Date: Fri, 22 May 2026 04:19:29 GMT
• Title: CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering
• Abstract summary: Cause-and-effect reasoning in video is a significant challenge for Vision-Language Models (VLMs)<n>We introduce CaST-Bench, a benchmark for Causal-Temporal Video Video Reasoning SST-Bench.<n>We construct a high-quality dataset of 2,066 questions over 1,015 videos, with causal chains annotated by temporal segments and bounding-box tracks.
• Score: 27.144152466342103
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Cause-and-effect reasoning in video is a significant challenge for Vision-Language Models (VLMs), as it requires going beyond surface-level perception to a deeper understanding of causal mechanisms. However, existing benchmarks rarely provide the fine-grained, grounded evidence needed to rigorously evaluate this capability. To address this gap, we introduce CaST-Bench, a benchmark for Causal Chain-Grounded Spatio-Temporal Video Reasoning. CaST-Bench presents complex causal questions that require models to identify and localize a chain of multiple spatio-temporal evidences. Through a human-AI collaborative pipeline, we construct a high-quality dataset of 2,066 questions over 1,015 videos, with causal chains annotated by temporal segments and bounding-box tracks. Furthermore, we design a comprehensive evaluation suite with novel metrics that assess not only answer correctness but also the capability for visual evidence grounded reasoning. This grounding is crucial for improving accuracy by mitigating spurious correlations and for enhancing user trust by making models more transparent. Our experiments show that current VLMs struggle with causal questions, largely due to their limited ability to construct precise and grounded causal chains. This highlights an important direction for improving future VLMs.

Related papers

• Evidence-Backed Video Question Answering [90.93225758130426]
  Existing explainability efforts rely on textual rationales or sparse bounding boxes.<n>We propose Evidence-Backed Video Question Answering (E-VQA)<n>To support this, we introduce ST-Evidence, the first human-verified benchmark for both discriminative and generative pixel-level grounding.
  arXiv  Detail & Related papers  (2026-07-13T17:49:10Z)
• Counterfactual Reasoning for Fine-Grained Evidence Disentanglement in VideoQA [16.471093160007587]
  VideoQA systems often rely on spurious statistical correlations rather than answer-relevant causal evidence.<n>Existing methods either rely on cross-modality correlations, costly curated training resources, or insufficient causal assumptions and constraints.<n>We propose a Counterfactual Reasoning framework for fine-grained Evidence Disentanglement (CREDiT)
  arXiv  Detail & Related papers  (2026-06-08T08:20:42Z)
• Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMs [49.55219052565761]
  CausalPhys is a benchmark of over 3,000 carefully curated video- and image-based questions spanning four domains: Perception, Anticipation, Intervention, and Goal Orientation.<n>Each question is paired with an expert-annotated causal graph capturing object-attribute-event dependencies, enabling interpretable and fine-grained evaluation of causal understanding.<n>Building on this, we formulate a causal-graph-grounded metric that quantitatively measures how well a model's chain-of-thought reasoning aligns with the correct causal relations.<n>We propose Causal Rationale-informed Fine-Tuning (CRFT), which explicitly
  arXiv  Detail & Related papers  (2026-06-04T10:07:05Z)
• Structured Causal Video Reasoning via Multi-Objective Alignment [102.61829546891543]
  We propose a compact representation of salient events and their causal relationships, which we name Structured Event Facts, prior to the reasoning stage.<n>This structured prior serves as an explicit constraint to promote concise and causally grounded reasoning.<n>We introduce CausalFact-60K and a four-stage training pipeline comprising facts alignment, format warm-start, thinking warm-start, and reinforcement learning-based post-training.
  arXiv  Detail & Related papers  (2026-04-06T04:49:30Z)
• VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification [73.02304272829785]
  VideoBench is a hierarchical benchmark for long-video answering that rigorously verifies evidence.<n>It comprises 500 manually annotated questions across 13 domains, paired with temporal intervals and spatial bounding boxes as evidence.<n>Experiments show that even GeminiPro-3-Pro correctly answers fewer than 17% of questions under the standard end-to-end QA setting.<n>Results expose a significant gap between surface-level answer correctness and genuine evidence-based reasoning.
  arXiv  Detail & Related papers  (2026-04-02T03:29:43Z)
• Video-R2: Reinforcing Consistent and Grounded Reasoning in Multimodal Language Models [56.851611990473174]
  Reasoning over dynamic visual content remains a central challenge for large language models.<n>We propose a reinforcement learning approach that enhances both temporal precision and reasoning consistency.<n>The resulting model, Video R2, achieves consistently higher TAC, VAS, and accuracy across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-11-28T18:59:58Z)
• ChainReaction! Structured Approach with Causal Chains as Intermediate Representations for Improved and Explainable Causal Video Question Answering [13.950975794272603]
  We propose a novel framework that explicitly decouples causal reasoning from answer generation.<n>Inspired by human cognitive models, structured cause-effect sequences bridge low-level video content with high-level causal reasoning.<n>We also propose CauCo, a new evaluation metric for causality-oriented captioning.
  arXiv  Detail & Related papers  (2025-08-28T17:10:53Z)
• VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models [29.706347050700867]
  We introduce a novel benchmark named Video-based long-form Causal Reasoning (VCRBench)<n>VCRBench tests whether Large Video Language Models (LVLMs) can identify, reason about, and correctly sequence the events needed to accomplish a specific goal.<n>We propose Recognition-Reasoning Decomposition (RRD), a modular approach that breaks video-based causal reasoning into two sub-tasks of video recognition and causal reasoning.
  arXiv  Detail & Related papers  (2025-05-13T11:35:58Z)
• Causality Model for Semantic Understanding on Videos [0.0]
  This thesis focuses on the domain of semantic video understanding.<n>It explores the potential of causal modeling to advance two fundamental tasks: Video Relation Detection (VidVRD) and Video Question Answering (VideoQA)
  arXiv  Detail & Related papers  (2025-03-16T10:44:11Z)
• V-STaR: Benchmarking Video-LLMs on Video Spatio-Temporal Reasoning [40.18308199837137]
  We introduce a Video S-Temporal Reasoning (V-STa) benchmark to address these shortcomings.<n>We construct a dataset to elicit the spatial-temporal reasoning process of Video-LLMs.<n>Experiments from 14 Video-LLMs reveal significant gaps between current Video-LLMs and the needs for robust and consistent consistent reasoning.
  arXiv  Detail & Related papers  (2025-03-14T15:21:44Z)
• Cross-modal Causal Relation Alignment for Video Question Grounding [44.97933293141372]
  Video question grounding (VideoQG) requires models to answer the questions and simultaneously infer the relevant video segments to support the answers.<n>Existing VideoQG methods usually suffer from spurious cross-modal correlations, leading to a failure to identify the dominant visual scenes that align with the intended question.<n>We propose a novel VideoQG framework named Cross-modal Causal Relation Alignment (CRA), to eliminate spurious correlations and improve the causal consistency between question-answering and video temporal grounding.
  arXiv  Detail & Related papers  (2025-03-05T01:36:32Z)
• STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training [87.58996020705258]
  Video Large Language Models (Video-LLMs) have recently shown strong derivation in basic video understanding tasks.<n>Video-LLMs struggle with compositional reasoning that requires multi-step explicit-temporal inference across object relations, interactions and events.<n>We propose STEP, a novel graph-guided self-training method that enables VideoLLMs to generate reasoning-rich finetuning data from any raw videos to improve itself.
  arXiv  Detail & Related papers  (2024-11-29T11:54:55Z)
• Invariant Grounding for Video Question Answering [72.87173324555846]
  Video Question Answering (VideoQA) is the task of answering questions about a video. In leading VideoQA models, the typical learning objective, empirical risk minimization (ERM), latches on superficial correlations between video-question pairs and answers. We propose a new learning framework, Invariant Grounding for VideoQA (IGV), to ground the question-critical scene.
  arXiv  Detail & Related papers  (2022-06-06T04:37:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.