Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation

• URL: http://arxiv.org/abs/2605.28230v1
• Date: Wed, 27 May 2026 09:44:18 GMT
• Title: Proprio: Latent Self-Scoring and Inference-Time Refinement for Physically Plausible Video Generation
• Abstract summary: Proprio is a training-free framework that enables a frozen generator to assess and improve the physical plausibility of its own outputs.<n>Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual as a self-scoring signal.<n>Proprio consistently improves physical plausibility across text-to-video and image-to-video benchmarks.
• Score: 78.41956766048308
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern video generative models produce visually impressive results, yet frequently violate basic physical principles. We propose Proprio, a training-free framework that enables a frozen video generator to assess and improve the physical plausibility of its own outputs. Inspired by proprioception, the biological sense of one's own movement, Proprio treats the model's flow residual under controlled latent perturbations as a self-scoring signal. Samples that are better explained by the generator's learned dynamics induce smaller and more stable residuals. We aggregate this signal across timesteps and perturbations, focus it on motion-relevant regions with a dynamic spatiotemporal mask, and use it for best-of-N search, gradient-based self-refinement, or both. Across text-to-video and image-to-video benchmarks, Proprio consistently improves physical plausibility, outperforming VLM-based scoring, and external world-model baselines in several settings. With TurboWan2.2, Proprio improves Physics-IQ from 32.2 to 37.5 (+16.5%) and VideoPhy2-hard physical commonsense from 45.6 to 55.0 (+20.6%). Human evaluation further shows that raters prefer Proprio-selected or refined videos for physical plausibility in roughly two-thirds of comparisons. These results suggest that frozen video generators contain actionable internal signals for evaluating and improving the physical plausibility of their own outputs.

Related papers

• The Invisible Hand of Physics: When Video Diffusion Models Know More Than They Show [19.515022194108955]
  We show that physical plausibility is linearly decodable from diffusion transformer states across IntPhys and InfLevel.<n>Surprisingly, this signal is absent from the VAE latent input and emerges inside the denoising transformer itself.
  arXiv  Detail & Related papers  (2026-06-03T18:11:51Z)
• PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation [75.96085587438279]
  Existing video rewards primarily rely on 2D perceptual signals, without explicitly modeling the 3D body state, contact, and dynamics underlying articulated human motion.<n>We propose PhyMotion, a structured, fine-grained motion reward that grounds recovered 3D human trajectories in a physics simulator and evaluates motion quality along multiple dimensions of physical feasibility.<n>Experiments show that PhyMotion achieves stronger correlation with human judgments than existing reward formulations.
  arXiv  Detail & Related papers  (2026-05-14T02:12:13Z)
• PhyCo: Learning Controllable Physical Priors for Generative Motion [55.59209981836171]
  We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation.<n>Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback.
  arXiv  Detail & Related papers  (2026-04-30T17:53:03Z)
• Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning [77.34919361116037]
  Video generation models are increasingly used as world simulators for storytelling, simulation, and embodied AI.<n>Existing evaluations largely rely on automated metrics or coarse human judgments such as preferences or rubric-based checks.<n>We introduce Physion-Eval, a benchmark of expert human reasoning for diagnosing physical realism failures in videos generated by five state-of-the-art models.
  arXiv  Detail & Related papers  (2026-03-20T03:25:41Z)
• Physical Simulator In-the-Loop Video Generation [96.87054314612142]
  Physical Simulator In-the-loop Video Generation (PSIVG) is a novel framework that integrates a physical simulator into the video diffusion process.<n>PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity.
  arXiv  Detail & Related papers  (2026-03-06T15:48:25Z)
• Self-Refining Video Sampling [91.0784344916165]
  We present self-refining video sampling, a simple method that uses a pre-trained video generator trained on large-scale datasets as its own self-refiner.<n> Experiments on state-of-the-art video generators demonstrate significant improvements in motion coherence and physics alignment.
  arXiv  Detail & Related papers  (2026-01-26T15:22:27Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• PhyMAGIC: Physical Motion-Aware Generative Inference with Confidence-guided LLM [17.554471769834453]
  We present PhyMAGIC, a training-free framework that generates physically consistent motion from a single image.<n>PhyMAGIC integrates a pre-trained image-to-video diffusion model, confidence-guided reasoning via LLMs, and a differentiable physics simulator.<n> Comprehensive experiments demonstrate that PhyMAGIC outperforms state-of-the-art video generators and physics-aware baselines.
  arXiv  Detail & Related papers  (2025-05-22T09:40:34Z)
• Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos [6.093379844890164]
  We propose a novel method to selectively incorporate the physics models with the kinematics observations in an online setting.<n>A recurrent neural network is introduced to realize a Kalman filter that attentively balances the kinematics input and simulated motion.<n>The proposed approach excels in the physics-based human pose estimation task and demonstrates the physical plausibility of the predictive dynamics.
  arXiv  Detail & Related papers  (2024-10-10T10:24:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.