Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments

• URL: http://arxiv.org/abs/2504.02918v1
• Date: Thu, 03 Apr 2025 15:21:17 GMT
• Title: Morpheus: Benchmarking Physical Reasoning of Video Generative Models with Real Physical Experiments
• Authors: Chenyu Zhang, Daniil Cherniavskii, Andrii Zadaianchuk, Antonios Tragoudaras, Antonios Vozikis, Thijmen Nijdam, Derck W. E. Prinzhorn, Mark Bodracska, Nicu Sebe, Efstratios Gavves,
• Abstract summary: We introduce Morpheus, a benchmark for evaluating video generation models on physical reasoning.<n>It features 80 real-world videos capturing physical phenomena, guided by conservation laws.<n>Our findings reveal that even with advanced prompting and video conditioning, current models struggle to encode physical principles.
• Score: 55.465371691714296
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent advances in image and video generation raise hopes that these models possess world modeling capabilities, the ability to generate realistic, physically plausible videos. This could revolutionize applications in robotics, autonomous driving, and scientific simulation. However, before treating these models as world models, we must ask: Do they adhere to physical conservation laws? To answer this, we introduce Morpheus, a benchmark for evaluating video generation models on physical reasoning. It features 80 real-world videos capturing physical phenomena, guided by conservation laws. Since artificial generations lack ground truth, we assess physical plausibility using physics-informed metrics evaluated with respect to infallible conservation laws known per physical setting, leveraging advances in physics-informed neural networks and vision-language foundation models. Our findings reveal that even with advanced prompting and video conditioning, current models struggle to encode physical principles despite generating aesthetically pleasing videos. All data, leaderboard, and code are open-sourced at our project page.

Related papers

• VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness [76.16523963623537]
  We introduce VBench-2.0, a benchmark designed to evaluate video generative models for intrinsic faithfulness. VBench-2.0 assesses five key dimensions: Human Fidelity, Controllability, Creativity, Physics, and Commonsense. By pushing beyond superficial faithfulness toward intrinsic faithfulness, VBench-2.0 aims to set a new standard for the next generation of video generative models.
  arXiv  Detail & Related papers  (2025-03-27T17:57:01Z)
• Synthetic Video Enhances Physical Fidelity in Video Synthesis [25.41774228022216]
  We investigate how to enhance the physical fidelity of video generation models by leveraging synthetic videos derived from computer graphics pipelines. We propose a solution that curates and integrates synthetic data while introducing a method to transfer its physical realism to the model. Our work offers one of the first empirical demonstrations that synthetic video enhances physical fidelity in video synthesis.
  arXiv  Detail & Related papers  (2025-03-26T00:45:07Z)
• VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation [66.58048825989239]
  VideoPhy-2 is an action-centric dataset for evaluating physical commonsense in generated videos.<n>We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos.<n>Our findings reveal major shortcomings, with even the best model achieving only 22% joint performance.
  arXiv  Detail & Related papers  (2025-03-09T22:49:12Z)
• Generative Physical AI in Vision: A Survey [78.07014292304373]
  Gene Artificial Intelligence (AI) has rapidly advanced the field of computer vision by enabling machines to create and interpret visual data with unprecedented sophistication. This transformation builds upon a foundation of generative models to produce realistic images, videos, and 3D/4D content. As generative models evolve to increasingly integrate physical realism and dynamic simulation, their potential to function as "world simulators" expands.
  arXiv  Detail & Related papers  (2025-01-19T03:19:47Z)
• Do generative video models understand physical principles? [15.534227431706773]
  AI video generation is undergoing a revolution, with quality and realism advancing rapidly.<n>Do video models learn "world models" that discover laws of physics, or are they merely sophisticated pixel predictors that achieve visual realism without understanding the physical principles of reality?<n>We address this question by developing Physics-IQ, a benchmark dataset that can only be solved by acquiring a deep understanding of various physical principles.
  arXiv  Detail & Related papers  (2025-01-14T20:59:37Z)
• Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation [51.750634349748736]
  Text-to-video (T2V) models have made significant strides in visualizing complex prompts. However, the capacity of these models to accurately represent intuitive physics remains largely unexplored. We introduce PhyGenBench to evaluate physical commonsense correctness in T2V generation.
  arXiv  Detail & Related papers  (2024-10-07T17:56:04Z)
• PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation [29.831214435147583]
  We present PhysGen, a novel image-to-video generation method. It produces a realistic, physically plausible, and temporally consistent video. Our key insight is to integrate model-based physical simulation with a data-driven video generation process.
  arXiv  Detail & Related papers  (2024-09-27T17:59:57Z)
• VideoPhy: Evaluating Physical Commonsense for Video Generation [93.28748850301949]
  We present VideoPhy, a benchmark designed to assess whether the generated videos follow physical commonsense for real-world activities. We then generate videos conditioned on captions from diverse state-of-the-art text-to-video generative models. Our human evaluation reveals that the existing models severely lack the ability to generate videos adhering to the given text prompts.
  arXiv  Detail & Related papers  (2024-06-05T17:53:55Z)
• DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors.<n>We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.