Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation

• URL: http://arxiv.org/abs/2508.10858v1
• Date: Thu, 14 Aug 2025 17:30:37 GMT
• Title: Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generation
• Authors: Harold Haodong Chen, Haojian Huang, Qifeng Chen, Harry Yang, Ser-Nam Lim,
• Abstract summary: PhysHPO is a novel framework for Hierarchical Cross-Modal Direct Preference Optimization.<n>It enables fine-grained preference alignment for physically plausible video generation.<n>We show that PhysHPO significantly improves physical plausibility and overall video generation quality of advanced models.
• Score: 80.89133198952187
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advancements in video generation have enabled the creation of high-quality, visually compelling videos. However, generating videos that adhere to the laws of physics remains a critical challenge for applications requiring realism and accuracy. In this work, we propose PhysHPO, a novel framework for Hierarchical Cross-Modal Direct Preference Optimization, to tackle this challenge by enabling fine-grained preference alignment for physically plausible video generation. PhysHPO optimizes video alignment across four hierarchical granularities: a) Instance Level, aligning the overall video content with the input prompt; b) State Level, ensuring temporal consistency using boundary frames as anchors; c) Motion Level, modeling motion trajectories for realistic dynamics; and d) Semantic Level, maintaining logical consistency between narrative and visuals. Recognizing that real-world videos are the best reflections of physical phenomena, we further introduce an automated data selection pipeline to efficiently identify and utilize "good data" from existing large-scale text-video datasets, thereby eliminating the need for costly and time-intensive dataset construction. Extensive experiments on both physics-focused and general capability benchmarks demonstrate that PhysHPO significantly improves physical plausibility and overall video generation quality of advanced models. To the best of our knowledge, this is the first work to explore fine-grained preference alignment and data selection for video generation, paving the way for more realistic and human-preferred video generation paradigms.

Related papers

• PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Models [100.65199317765608]
  Physical principles are fundamental to realistic visual simulation, but remain a significant oversight in transformer-based video generation.<n>We introduce a physics-aware reinforcement learning paradigm for video generation models that enforces physical collision rules directly in high-dimensional spaces.<n>We extend this paradigm to a unified framework, termed Mimicry-Discovery Cycle (MDcycle), which allows substantial fine-tuning.
  arXiv  Detail & Related papers  (2026-01-16T08:40:10Z)
• PhysCorr: Dual-Reward DPO for Physics-Constrained Text-to-Video Generation with Automated Preference Selection [10.498184571108995]
  We propose PhysCorr, a unified framework for modeling, evaluating, and optimizing physical consistency in video generation.<n>Specifically, we introduce PhysicsRM, the first dual-dimensional reward model that quantifies both intra-object stability and inter-object interactions.<n>Our approach is model-agnostic and scalable, enabling seamless integration into a wide range of video diffusion and transformer-based backbones.
  arXiv  Detail & Related papers  (2025-11-06T02:40:57Z)
• RDPO: Real Data Preference Optimization for Physics Consistency Video Generation [24.842288734103505]
  We present Real Data Preference Optimisation (RDPO), an annotation-free framework that distills physical priors directly from real-world videos.<n>RDPO reverse-samples real video sequences with a pre-trained generator to automatically build preference pairs that are distinguishable in terms of physical correctness.<n>A multi-stage iterative training schedule guides the generator to obey physical laws increasingly well.
  arXiv  Detail & Related papers  (2025-06-23T13:55:24Z)
• Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation [41.17844925831194]
  Vid2Sim is a generalizable video-based approach for recovering geometry and physical properties.<n>A feed-forward neural network trained to capture physical world knowledge reconstructs the observed configuration of the physical system from video.<n>A lightweight optimization pipeline refines the estimated appearance, geometry, and physical properties to closely align with video observations.
  arXiv  Detail & Related papers  (2025-06-06T18:00:46Z)
• Motion aware video generative model [12.5036873986483]
  Diffusion-based video generation has yielded unprecedented quality in visual content and semantic coherence.<n>Current approaches rely on statistical learning without explicitly modeling the underlying physics of motion.<n>This paper introduces a physics-informed frequency domain approach to enhance the physical plausibility of generated videos.
  arXiv  Detail & Related papers  (2025-06-02T20:42:54Z)
• RAGME: Retrieval Augmented Video Generation for Enhanced Motion Realism [73.38167494118746]
  We propose a framework to improve the realism of motion in generated videos.<n>We advocate for the incorporation of a retrieval mechanism during the generation phase.<n>Our pipeline is designed to apply to any text-to-video diffusion model.
  arXiv  Detail & Related papers  (2025-04-09T08:14:05Z)
• VPO: Aligning Text-to-Video Generation Models with Prompt Optimization [80.86205966195593]
  Video generation models are typically trained on text-to-video pairs with highly detailed and carefully crafted descriptions.<n>We introduce VPO, a principled framework that optimize prompts based on three core principles: harmlessness, accuracy, and helpfulness.<n>Our experiments demonstrate that VPO significantly improves safety, alignment, and video quality compared to baseline methods.
  arXiv  Detail & Related papers  (2025-03-26T12:28:20Z)
• Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback [130.090296560882]
  We investigate the use of feedback to enhance the object dynamics in text-to-video models.<n>We show that our method can effectively optimize a wide variety of rewards, with binary AI feedback driving the most significant improvements in video quality for dynamic interactions.
  arXiv  Detail & Related papers  (2024-12-03T17:44:23Z)
• VideoPhy: Evaluating Physical Commonsense for Video Generation [93.28748850301949]
  We present VideoPhy, a benchmark designed to assess whether the generated videos follow physical commonsense for real-world activities. We then generate videos conditioned on captions from diverse state-of-the-art text-to-video generative models. Our human evaluation reveals that the existing models severely lack the ability to generate videos adhering to the given text prompts.
  arXiv  Detail & Related papers  (2024-06-05T17:53:55Z)
• E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning [53.63364311738552]
  Bio-inspired event cameras or dynamic vision sensors are capable of capturing per-pixel brightness changes (called event-streams) in high temporal resolution and high dynamic range. It calls for events-to-video (E2V) solutions which take event-streams as input and generate high quality video frames for intuitive visualization. We propose textbfE2HQV, a novel E2V paradigm designed to produce high-quality video frames from events.
  arXiv  Detail & Related papers  (2024-01-16T05:10:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.