Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception

• URL: http://arxiv.org/abs/2411.16800v4
• Date: Sun, 13 Apr 2025 07:07:04 GMT
• Title: Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception
• Authors: Jiajing Lin, Zhenzhong Wang, Dejun Xu, Shu Jiang, YunPeng Gong, Min Jiang,
• Abstract summary: Phys4DGen is a novel 4D generation framework that integrates multi-material composition perception with physical simulation.<n>The framework achieves automated, physically plausible 4D generation through three innovative modules.<n> Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism.
• Score: 4.054634170768821
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 4D content generation aims to create dynamically evolving 3D content that responds to specific input objects such as images or 3D representations. Current approaches typically incorporate physical priors to animate 3D representations, but these methods suffer from significant limitations: they not only require users lacking physics expertise to manually specify material properties but also struggle to effectively handle the generation of multi-material composite objects. To address these challenges, we propose Phys4DGen, a novel 4D generation framework that integrates multi-material composition perception with physical simulation. The framework achieves automated, physically plausible 4D generation through three innovative modules: first, the 3D Material Grouping module partitions heterogeneous material regions on 3D representation surfaces via semantic segmentation; second, the Internal Physical Structure Discovery module constructs the mechanical structure of object interiors; finally, we distill physical prior knowledge from multimodal large language models to enable rapid and automatic material properties identification for both objects' surfaces and interiors. Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism in open-world scenarios, significantly outperforming state-of-the-art methods.

Related papers

• SOPHY: Generating Simulation-Ready Objects with Physical Materials [10.156212838002903]
  SOPHY is a generative model for 3D physics-aware shape synthesis. Our approach jointly synthesizes shape, texture, and material properties related to physics-grounded dynamics.
  arXiv  Detail & Related papers  (2025-04-17T06:17:24Z)
• OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation [25.790371838717284]
  We propose OmniPhysGS for synthesizing a physics-based 3D dynamic scene composed of more general objects. Our method surpasses existing methods by approximately 3% to 16% in metrics of visual quality and text alignment.
  arXiv  Detail & Related papers  (2025-01-31T09:28:07Z)
• Text-to-3D Gaussian Splatting with Physics-Grounded Motion Generation [47.6666060652434]
  We present an innovative framework that generates 3D models with accurate appearances and geometric structures. By integrating text-to-3D generation with physics-grounded motion synthesis, our framework renders photo-realistic 3D objects.
  arXiv  Detail & Related papers  (2024-12-07T06:48:16Z)
• PhysMotion: Physics-Grounded Dynamics From a Single Image [24.096925413047217]
  We introduce PhysMotion, a novel framework that leverages principled physics-based simulations to guide intermediate 3D representations generated from a single image. Our approach addresses the limitations of traditional data-driven generative models and result in more consistent physically plausible motions.
  arXiv  Detail & Related papers  (2024-11-26T07:59:11Z)
• PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generation [29.831214435147583]
  We present PhysGen, a novel image-to-video generation method. It produces a realistic, physically plausible, and temporally consistent video. Our key insight is to integrate model-based physical simulation with a data-driven video generation process.
  arXiv  Detail & Related papers  (2024-09-27T17:59:57Z)
• Phy124: Fast Physics-Driven 4D Content Generation from a Single Image [3.0613673973976625]
  We introduce Phy124, a novel, fast, and physics-driven method for controllable 4D content generation from a single image. Phy124 integrates physical simulation directly into the 4D generation process, ensuring the resulting 4D content adheres to natural physical laws. Experiments demonstrate that Phy124 generates high-fidelity 4D content with significantly reduced inference times.
  arXiv  Detail & Related papers  (2024-09-11T10:41:46Z)
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion [35.71595369663293]
  We propose textbfPhysics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model. Experiments demonstrate the effectiveness of our method with both elastic and plastic materials.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• DreamPhysics: Learning Physical Properties of Dynamic 3D Gaussians with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors. We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)
• Physically Compatible 3D Object Modeling from a Single Image [109.98124149566927]
  We present a framework that transforms single images into 3D physical objects. Our framework embeds physical compatibility into the reconstruction process. It consistently enhances the physical realism of 3D models over existing methods.
  arXiv  Detail & Related papers  (2024-05-30T21:59:29Z)
• Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models [116.31344506738816]
  We present a novel framework, textbfDiffusion4D, for efficient and scalable 4D content generation. We develop a 4D-aware video diffusion model capable of synthesizing orbital views of dynamic 3D assets. Our method surpasses prior state-of-the-art techniques in terms of generation efficiency and 4D geometry consistency.
  arXiv  Detail & Related papers  (2024-05-26T17:47:34Z)
• Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials [108.59709545364395]
  GPT-4V can effectively recognize and describe materials, allowing the construction of a detailed material library. The correctly matched materials are then meticulously applied as reference for the new SVBRDF material generation. Make-it-Real offers a streamlined integration into the 3D content creation workflow.
  arXiv  Detail & Related papers  (2024-04-25T17:59:58Z)
• PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation [62.53760963292465]
  PhysDreamer is a physics-based approach that endows static 3D objects with interactive dynamics. We present our approach on diverse examples of elastic objects and evaluate the realism of the synthesized interactions through a user study.
  arXiv  Detail & Related papers  (2024-04-19T17:41:05Z)
• SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer [57.506654943449796]
  We propose an efficient, sparse-controlled video-to-4D framework named SC4D that decouples motion and appearance. Our method surpasses existing methods in both quality and efficiency. We devise a novel application that seamlessly transfers motion onto a diverse array of 4D entities.
  arXiv  Detail & Related papers  (2024-04-04T18:05:18Z)
• D&D: Learning Human Dynamics from Dynamic Camera [55.60512353465175]
  We present D&D (Learning Human Dynamics from Dynamic Camera), which leverages the laws of physics to reconstruct 3D human motion from the in-the-wild videos with a moving camera. Our approach is entirely neural-based and runs without offline optimization or simulation in physics engines.
  arXiv  Detail & Related papers  (2022-09-19T06:51:02Z)
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation [75.0278287071591]
  ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science.
  arXiv  Detail & Related papers  (2020-07-09T17:33:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.