Sora Generates Videos with Stunning Geometrical Consistency
- URL: http://arxiv.org/abs/2402.17403v1
- Date: Tue, 27 Feb 2024 10:49:05 GMT
- Title: Sora Generates Videos with Stunning Geometrical Consistency
- Authors: Xuanyi Li, Daquan Zhou, Chenxu Zhang, Shaodong Wei, Qibin Hou and
Ming-Ming Cheng
- Abstract summary: We introduce a new benchmark that assesses the quality of the generated videos based on their adherence to real-world physics principles.
We employ a method that transforms the generated videos into 3D models, leveraging the premise that the accuracy of 3D reconstruction is heavily contingent on the video quality.
- Score: 75.46675626542837
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The recently developed Sora model [1] has exhibited remarkable capabilities
in video generation, sparking intense discussions regarding its ability to
simulate real-world phenomena. Despite its growing popularity, there is a lack
of established metrics to evaluate its fidelity to real-world physics
quantitatively. In this paper, we introduce a new benchmark that assesses the
quality of the generated videos based on their adherence to real-world physics
principles. We employ a method that transforms the generated videos into 3D
models, leveraging the premise that the accuracy of 3D reconstruction is
heavily contingent on the video quality. From the perspective of 3D
reconstruction, we use the fidelity of the geometric constraints satisfied by
the constructed 3D models as a proxy to gauge the extent to which the generated
videos conform to real-world physics rules. Project page:
https://sora-geometrical-consistency.github.io/
Related papers
- What Matters in Detecting AI-Generated Videos like Sora? [51.05034165599385]
Gap between synthetic and real-world videos remains under-explored.
In this study, we compare real-world videos with those generated by a state-of-the-art AI model, Stable Video Diffusion.
Our model is capable of detecting videos generated by Sora with high accuracy, even without exposure to any Sora videos during training.
arXiv Detail & Related papers (2024-06-27T23:03:58Z) - DreamPhysics: Learning Physical Properties of Dynamic 3D Gaussians with Video Diffusion Priors [77.34056839349076]
We propose DreamPhysics, which estimates physical properties of 3D Gaussian Splatting with video diffusion priors.
Based on a material point method simulator with proper physical parameters, our method can generate 4D content with realistic motions.
arXiv Detail & Related papers (2024-06-03T16:05:25Z) - Precise-Physics Driven Text-to-3D Generation [24.180947937863355]
We propose Phy3DGen, a precise-physics-driven text-to-3D generation method.
By analyzing the solid mechanics of generated 3D shapes, we reveal that the 3D shapes generated by existing text-to-3D generation methods are impractical for real-world applications.
arXiv Detail & Related papers (2024-03-19T04:51:38Z) - Towards Live 3D Reconstruction from Wearable Video: An Evaluation of
V-SLAM, NeRF, and Videogrammetry Techniques [20.514826446476267]
Mixed reality (MR) is a key technology which promises to change the future of warfare.
To enable this technology, a large-scale 3D model of a physical environment must be maintained based on live sensor observations.
We survey several 3D reconstruction algorithms for large-scale mapping for military applications given only live video.
arXiv Detail & Related papers (2022-11-21T19:57:51Z) - 3D-Aware Video Generation [149.5230191060692]
We explore 4D generative adversarial networks (GANs) that learn generation of 3D-aware videos.
By combining neural implicit representations with time-aware discriminator, we develop a GAN framework that synthesizes 3D video supervised only with monocular videos.
arXiv Detail & Related papers (2022-06-29T17:56:03Z) - LASR: Learning Articulated Shape Reconstruction from a Monocular Video [97.92849567637819]
We introduce a template-free approach to learn 3D shapes from a single video.
Our method faithfully reconstructs nonrigid 3D structures from videos of human, animals, and objects of unknown classes.
arXiv Detail & Related papers (2021-05-06T21:41:11Z) - Online Adaptation for Consistent Mesh Reconstruction in the Wild [147.22708151409765]
We pose video-based reconstruction as a self-supervised online adaptation problem applied to any incoming test video.
We demonstrate that our algorithm recovers temporally consistent and reliable 3D structures from videos of non-rigid objects including those of animals captured in the wild.
arXiv Detail & Related papers (2020-12-06T07:22:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.