PhyCo: Learning Controllable Physical Priors for Generative Motion

• URL: http://arxiv.org/abs/2604.28169v1
• Date: Thu, 30 Apr 2026 17:53:03 GMT
• Title: PhyCo: Learning Controllable Physical Priors for Generative Motion
• Abstract summary: We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
• Score: 55.59209981836171
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern video diffusion models excel at appearance synthesis but still struggle with physical consistency: objects drift, collisions lack realistic rebound, and material responses seldom match their underlying properties. We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation. Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback. This combination enables a generative model to produce physically consistent and controllable outputs through variations in physical attributes-without any simulator or geometry reconstruction at inference. On the Physics-IQ benchmark, PhyCo significantly improves physical realism over strong baselines, and human studies confirm clearer and more faithful control over physical attributes. Our results demonstrate a scalable path toward physically consistent, controllable generative video models that generalize beyond synthetic training environments.

Related papers

• Learning Explicit Physical Parameter Control and Benchmarking for Video Generation [21.331326205371628]
  We present PhyParam, a physics-guided image-to-video diffusion model that conditions on object-level forces, masses, friction, restitution, and scene-level gravity.<n>We also establish PhyParam-Bench, a benchmark for physical-law consistency in image-to-video generation.
  arXiv  Detail & Related papers  (2026-07-21T10:06:55Z)
• PhysAgent: Reflective Agentic Physics Control for Physically Plausible Video Generation [68.13035350819901]
  PhysAgent is a framework that closes the loop among physical program generation, physics simulation, stage-specific verification, and targeted program repair.<n>Our framework produces more physically plausible videos, achieves better prompt alignment, and generalizes more effectively across diverse physical scenarios.
  arXiv  Detail & Related papers  (2026-07-17T08:55:09Z)
• MMPhysVideo: Scaling Physical Plausibility in Video Generation via Joint Multimodal Modeling [55.72785604682579]
  MMPhysVideo is a framework to scale physical plausibility in video generation in joint multimodal modeling.<n>MMPhysVideo consistently improves physical plausibility and visual quality over advanced models across various benchmarks.
  arXiv  Detail & Related papers  (2026-04-03T07:32:24Z)
• PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignment [23.066204478030627]
  Video Models (VDMs) offer a promising approach for simulating dynamic scenes and environments.<n>Existing models often generate temporally incoherent content that violates basic physical intuition.<n>We propose PhysAlign, an efficient framework for physics-coherent image-to-videoI (2V) generation.
  arXiv  Detail & Related papers  (2026-03-14T05:44:18Z)
• PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models [100.65199317765608]
  Physical principles are fundamental to realistic visual simulation, but remain a significant oversight in transformer-based video generation.<n>We introduce a physics-aware reinforcement learning paradigm for video generation models that enforces physical collision rules directly in high-dimensional spaces.<n>We extend this paradigm to a unified framework, termed Mimicry-Discovery Cycle (MDcycle), which allows substantial fine-tuning.
  arXiv  Detail & Related papers  (2026-01-16T08:40:10Z)
• PhysChoreo: Physics-Controllable Video Generation with Part-Aware Semantic Grounding [50.454084539837005]
  PhysChoreo is a novel framework that can generate videos with diverse controllability and physical realism from a single image.<n>Our method consists of two stages: first, it estimates the static initial physical properties of all objects in the image through part-aware physical property reconstruction.<n>Then, through temporally instructed and physically editable simulation, it synthesizes high-quality videos with rich dynamic behaviors and physical realism.
  arXiv  Detail & Related papers  (2025-11-25T17:59:04Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference [57.086932851733145]
  We introduce LikePhys, a training-free method that evaluates intuitive physics in video diffusion models.<n>We benchmark intuitive physics understanding in current video diffusion models.<n> Empirical results show that, despite current models struggling with complex and chaotic dynamics, there is a clear trend of improvement in physics understanding as model capacity and inference settings scale.
  arXiv  Detail & Related papers  (2025-10-13T15:19:07Z)
• PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation [53.06495362038348]
  Existing generation models excel at producing photo-realistic videos from text or images, but often lack physical plausibility and 3D controllability.<n>We introduce PhysCtrl, a novel framework for physics-grounded image-to-video generation with physical parameters and force control.<n> Experiments show that PhysCtrl generates realistic, physics-grounded motion trajectories which, when used to drive image-to-video models, yield high-fidelity, controllable videos.
  arXiv  Detail & Related papers  (2025-09-24T17:58:04Z)
• PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided LLM [17.554471769834453]
  We present PhyMAGIC, a training-free framework that generates physically consistent motion from a single image.<n>PhyMAGIC integrates a pre-trained image-to-video diffusion model, confidence-guided reasoning via LLMs, and a differentiable physics simulator.<n> Comprehensive experiments demonstrate that PhyMAGIC outperforms state-of-the-art video generators and physics-aware baselines.
  arXiv  Detail & Related papers  (2025-05-22T09:40:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.