CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation

• URL: http://arxiv.org/abs/2606.14317v1
• Date: Fri, 12 Jun 2026 09:57:01 GMT
• Title: CausalMotion: Structured Physical Reasoning as Keyframe and Trajectory Guidance for Training-Free Video Generation
• Abstract summary: textbfCausalMotion injects explicit physical reasoning into video generation through structured intermediate representations.<n>Our method consistently improves physical plausibility and temporal coherence, particularly in dynamics-intensive scenarios.
• Score: 31.482087672315895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in diffusion-based video generation have significantly improved visual quality and short-term temporal coherence. However, existing methods still struggle to produce videos with physically consistent and causally plausible dynamics, especially in scenarios involving long-horizon interactions. This limitation arises from the fact that video diffusion models primarily learn physical consistency implicitly, while vision-language models can directly model physical laws. Based on this idea, in this work, we propose \textbf{CausalMotion}, a training-free framework that injects explicit physical reasoning into video generation through structured intermediate representations. Our key idea is to decouple reasoning from generation by leveraging a vision-language model to decompose a text prompt into a sequence of causally consistent keyframes and object-centric motion trajectories. These representations are then aligned and integrated as soft constraints to guide a pretrained video diffusion model during inference. This design enables explicit modeling of object dynamics and causal transitions without requiring additional training or supervision. Extensive experiments show that our method consistently improves physical plausibility and temporal coherence, particularly in dynamics-intensive scenarios, while maintaining high perceptual video quality.

Related papers

• Self-Supervised Learning of Structured Dynamics from Videos [76.23834498074805]
  We study whether structured motion representations can be recovered from frozen features of a pretrained image vision transformer.<n>We propose the Structured Dynamics Model (SDM), which explicitly separates the dominant source of temporal change from residual dynamics.<n>We evaluate SDM on ProbeMotion, a new evaluation suite spanning synthetic and real videos with camera motion, object motion, and combined dynamics.
  arXiv  Detail & Related papers  (2026-07-23T17:55:07Z)
• SIFT: Self-Imagination Fine-Tuning for Physically Plausible Motion in Video Diffusion Models [53.82339966194578]
  Motion entanglement is the unintended coupling of independent motion sources, such as camera movement and object motion.<n>We propose a Self-Imagination Fine-Tuning (SIFT) paradigm, which enables the model to learn from its own generated videos.
  arXiv  Detail & Related papers  (2026-06-26T05:39:40Z)
• Tempered Self-Similarity Alignment for Physically Plausible Video Generation [83.40337664171939]
  Self-similarity loss transforms into probabilistic correspondence and trains the video generative model to align its correspondence with those of the visual foundation model on dynamically changing regions.<n>Our method demonstrates substantial improvements in physical plausibility across diverse interaction scenarios, validating the effectiveness of transferring relational knowledge for video generation.
  arXiv  Detail & Related papers  (2026-05-24T09:28:05Z)
• Demystifing Video Reasoning [71.53763299316041]
  We show that reasoning in video models instead primarily emerges along the diffusion denoising steps.<n>We identify several emergent reasoning behaviors critical to model performance.<n>Motivated by these insights, we present a training-free strategy as a proof-of-concept.
  arXiv  Detail & Related papers  (2026-03-17T17:59:55Z)
• PhyRPR: Training-Free Physics-Constrained Video Generation [12.488633253476111]
  We propose a training-free three-stage pipeline for video generation.<n>textitPhyRPR:textitPhyulineReason--textitPhyulinePlan--textitPhyulineRefine.<n>textitPhyReason uses a large multimodal model for physical state reasoning and an image generator for synthesis.<n>textitPhyPlan deterministically synthesizes a controllable coarse motion scaffold.<n>textit
  arXiv  Detail & Related papers  (2026-01-14T07:41:56Z)
• Show Me: Unifying Instructional Image and Video Generation with Diffusion Models [16.324312147741495]
  We propose a unified framework that enables image manipulation and video prediction.<n>We introduce structure and motion consistency rewards to improve structural fidelity and temporal coherence.<n> Experiments on diverse benchmarks demonstrate that our method outperforms expert models in both instructional image and video generation.
  arXiv  Detail & Related papers  (2025-11-21T23:24:28Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• Physics-Guided Motion Loss for Video Generation Model [8.083315267770255]
  Current video diffusion models generate visually compelling content but often violate basic laws of physics.<n>We introduce a frequency-domain physics prior that improves motion plausibility without modifying model architectures.
  arXiv  Detail & Related papers  (2025-06-02T20:42:54Z)
• Think Before You Diffuse: Infusing Physical Rules into Video Diffusion [55.046699347579455]
  The complexity of real-world motions, interactions, and dynamics introduce great difficulties when learning physics from data.<n>We propose DiffPhy, a generic framework that enables physically-correct and photo-realistic video generation by fine-tuning a pre-trained video diffusion model.
  arXiv  Detail & Related papers  (2025-05-27T18:26:43Z)
• VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Prior [88.51778468222766]
  Video diffusion models (VDMs) have advanced significantly in recent years, enabling the generation of highly realistic videos.<n>VDMs often fail to produce physically plausible videos due to an inherent lack of understanding of physics.<n>We propose a novel two-stage image-to-video generation framework that explicitly incorporates physics with vision and language informed physical prior.
  arXiv  Detail & Related papers  (2025-03-30T09:03:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.