Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning

• URL: http://arxiv.org/abs/2605.21988v1
• Date: Thu, 21 May 2026 04:38:02 GMT
• Title: Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning
• Abstract summary: Video large language models (Video models) achieve strong benchmark accuracy, yet often answer video questions through shortcuts such as single-frame cues and language priors rather than by tracking dynamics.<n>This issue is exacerbated in RL post-training, where correctness-only rewards can further reinforce shortcut policies that obtain high reward without tracking video dynamics.<n>We address this by asking a controlled counterfactual question: if the visual world changed while the answer remained fixed, should the answer change or stay the same?
• Score: 24.29101453473451
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video large language models (Video LLMs) achieve strong benchmark accuracy, yet often answer video questions through shortcuts such as single-frame cues and language priors rather than by tracking spatiotemporal dynamics. This issue is exacerbated in RL post-training, where correctness-only rewards can further reinforce shortcut policies that obtain high reward without tracking video dynamics. We address this by asking a controlled counterfactual question: if the visual world changed while the question remained fixed, should the answer change or stay the same? Based on this view, we propose \textbf{Counterfactual Relational Policy Optimization (CRPO)}, a dual-branch RL framework for improving \emph{spatiotemporal sensitivity}. CRPO constructs counterfactual videos through horizontal flips and temporal reversals, trains on both original and counterfactual branches, and introduces a \textbf{Counterfactual Relation Reward (CRR)} between their answers. CRR encourages answers to change for dynamic questions and remain unchanged for static questions. This cross-branch constraint makes it difficult for shortcut policies to be consistently rewarded across both branches. To evaluate this property, we introduce \textbf{DyBench}, a paired counterfactual video benchmark with 3,014 videos covering reversible dynamics, moving direction, and event sequence, together with a strict pair-accuracy metric that prevents fixed-answer shortcuts from inflating scores. Experiments show that CRPO outperforms prior RL methods on spatiotemporal-sensitive evaluations while maintaining competitive general video performance. On Qwen3-VL-8B, CRPO improves DyBench P-Acc by +7.7 and TimeBlind I-Acc by +8.2 over the base model, indicating improved spatiotemporal sensitivity rather than stronger reliance on static shortcuts. The project website can be found at https://ddz16.github.io/crpo.github.io/ .

Related papers

• Intrinsic Temporal Adaptation of CLIP for Partially Relevant Video Retrieval [40.983482307220676]
  PRVR aims to retrieve untrimmed videos that contain moments relevant to a text query.<n>We propose an Intrinsic Temporal Adaptation (ITA) framework for PRVR.<n>Our method achieves state-of-the-art performance on PRVR benchmarks.
  arXiv  Detail & Related papers  (2026-09-04T06:58:59Z)
• Archer: Adaptive Reuse of Cached Hidden States for Efficient Rollback in Diffusion Language Models [28.20034213862415]
  Diffusion language models (DLMs) iteratively refine a sequence, allowing earlier predictions to be revised as context evolves.<n>Every denoising update alters the global context, forcing both prompt and response states to be recomputed.<n>We introduce Adaptive Reuse of Cached Hidden States for Efficient Rollback (Archer)<n>Archer is a training-free KV caching method for rollback-capable DLMs.
  arXiv  Detail & Related papers  (2026-08-08T12:13:10Z)
• video-SALMONN-R$^3$: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding [49.5110151334134]
  Video large language models (LLMs) are often constrained by computation and memory budgets.<n>Video-SALMONN-R$3$ is the first end-to-end video-LLM that enables re-watch through reinforcement learning.
  arXiv  Detail & Related papers  (2026-06-23T12:13:19Z)
• Find, Fix, Reason: Context Repair for Video Reasoning [45.021693494492666]
  Reinforcement learning has advanced video reasoning in large multi-modal models.<n>We introduce an observation-level intervention: a frozen, tool-integrated teacher identifies the missingtemporal dependency and provides a minimal evidence patch.<n>We propose a Robust Improvement Reward (RIR) that aligns optimization with two goals: outcome validity through correct answers and dependency alignment rationales.
  arXiv  Detail & Related papers  (2026-04-17T17:04:19Z)
• TemporalDoRA: Temporal PEFT for Robust Surgical Video Question Answering [12.77631688698326]
  We introduce TemporalDoRA, a video-specific PEFT formulation that extends Weight-Decomposed Low-Rank Adaptation.<n>By mixing information across frames within the adaptation subspace, TemporalDoRA steers updates toward temporally consistent visual cues.
  arXiv  Detail & Related papers  (2026-03-10T14:03:42Z)
• TV-RAG: A Temporal-aware and Semantic Entropy-Weighted Framework for Long Video Retrieval and Understanding [14.570869250170139]
  TV-RAG is a training-free architecture that couples temporal alignment with entropy-guided semantics to improve long-video reasoning.<n>By weaving these temporal and semantic signals together, TV-RAG realises a dual-level reasoning routine that can be grafted onto any LVLM without re-training or fine-tuning.
  arXiv  Detail & Related papers  (2025-12-29T14:10:22Z)
• Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-in [80.03914556721519]
  Grounded video question answering (GVQA) aims to localize relevant temporal segments in videos and generate accurate answers to a given question.<n>We present Zoom-Zero, a framework that first localizes query-relevant segments and then temporally zooms into the most salient frames for finer-grained visual verification.
  arXiv  Detail & Related papers  (2025-12-16T10:34:39Z)
• Video-LLMs with Temporal Visual Screening [59.18455762289321]
  Temporal Visual Screening (TVS) is a new task that universally pre-processes video question answering and instruction tuning data.<n>TVS is formulated as a modular front-end adapter task that can be seamlessly integrated into both Video Instruction Tuning (training) and Video Question Answering (inference) pipelines.<n> Experiments demonstrate that incorporating TVS yields relative gains of 7.33% (training) and 34.6% (inference)
  arXiv  Detail & Related papers  (2025-08-27T14:33:32Z)
• Reinforcing Video Reasoning with Focused Thinking [65.85683941058916]
  We propose TW-GRPO, a novel framework that enhances visual reasoning with focused thinking and dense reward granularity.<n>Specifically, we employ a token weighting mechanism that prioritizes tokens with high informational density.<n>We also reformulate RL training by shifting from single-choice to multi-choice QA tasks.
  arXiv  Detail & Related papers  (2025-05-30T15:42:19Z)
• Transform-Equivariant Consistency Learning for Temporal Sentence Grounding [66.10949751429781]
  We introduce a novel Equivariant Consistency Regulation Learning framework to learn more discriminative representations for each video. Our motivation comes from that the temporal boundary of the query-guided activity should be consistently predicted. In particular, we devise a self-supervised consistency loss module to enhance the completeness and smoothness of the augmented video.
  arXiv  Detail & Related papers  (2023-05-06T19:29:28Z)
• Frame-Subtitle Self-Supervision for Multi-Modal Video Question Answering [73.11017833431313]
  Multi-modal video question answering aims to predict correct answer and localize the temporal boundary relevant to the question. We devise a weakly supervised question grounding (WSQG) setting, where only QA annotations are used. We transform the correspondence between frames and subtitles to Frame-Subtitle (FS) self-supervision, which helps to optimize the temporal attention scores.
  arXiv  Detail & Related papers  (2022-09-08T07:20:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.