OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation

• URL: http://arxiv.org/abs/2501.18982v1
• Date: Fri, 31 Jan 2025 09:28:07 GMT
• Title: OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation
• Authors: Yuchen Lin, Chenguo Lin, Jianjin Xu, Yadong Mu,
• Abstract summary: We propose OmniPhysGS for synthesizing a physics-based 3D dynamic scene composed of more general objects.<n>Our method surpasses existing methods by approximately 3% to 16% in metrics of visual quality and text alignment.
• Score: 25.790371838717284
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, significant advancements have been made in the reconstruction and generation of 3D assets, including static cases and those with physical interactions. To recover the physical properties of 3D assets, existing methods typically assume that all materials belong to a specific predefined category (e.g., elasticity). However, such assumptions ignore the complex composition of multiple heterogeneous objects in real scenarios and tend to render less physically plausible animation given a wider range of objects. We propose OmniPhysGS for synthesizing a physics-based 3D dynamic scene composed of more general objects. A key design of OmniPhysGS is treating each 3D asset as a collection of constitutive 3D Gaussians. For each Gaussian, its physical material is represented by an ensemble of 12 physical domain-expert sub-models (rubber, metal, honey, water, etc.), which greatly enhances the flexibility of the proposed model. In the implementation, we define a scene by user-specified prompts and supervise the estimation of material weighting factors via a pretrained video diffusion model. Comprehensive experiments demonstrate that OmniPhysGS achieves more general and realistic physical dynamics across a broader spectrum of materials, including elastic, viscoelastic, plastic, and fluid substances, as well as interactions between different materials. Our method surpasses existing methods by approximately 3% to 16% in metrics of visual quality and text alignment.

Related papers

• SOPHY: Generating Simulation-Ready Objects with Physical Materials [10.156212838002903]
  SOPHY is a generative model for 3D physics-aware shape synthesis. Our approach jointly synthesizes shape, texture, and material properties related to physics-grounded dynamics.
  arXiv  Detail & Related papers  (2025-04-17T06:17:24Z)
• GaussianProperty: Integrating Physical Properties to 3D Gaussians with LMMs [21.3615403516602]
  Estimating physical properties for visual data is a crucial task in computer vision, graphics, and robotics. We introduce GaussianProperty, a training-free framework that assigns physical properties of materials to 3D Gaussians. We demonstrate that 3D Gaussians with physical property annotations enable applications in physics-based dynamic simulation and robotic grasping.
  arXiv  Detail & Related papers  (2024-12-15T17:44:10Z)
• Phys4DGen: Physics-Compliant 4D Generation with Multi-Material Composition Perception [4.054634170768821]
  Phys4DGen is a novel 4D generation framework that integrates multi-material composition perception with physical simulation. The framework achieves automated, physically plausible 4D generation through three innovative modules. Experiments on both synthetic and real-world datasets demonstrate that Phys4DGen can generate high-fidelity 4D content with physical realism.
  arXiv  Detail & Related papers  (2024-11-25T12:12:38Z)
• Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
  We introduce latent intuitive physics, a transfer learning framework for physics simulation. It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes. We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
  arXiv  Detail & Related papers  (2024-06-18T16:37:44Z)
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion [35.71595369663293]
  We propose textbfPhysics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model. Experiments demonstrate the effectiveness of our method with both elastic and plastic materials.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors.<n>We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)
• PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation [62.53760963292465]
  PhysDreamer is a physics-based approach that endows static 3D objects with interactive dynamics. We present our approach on diverse examples of elastic objects and evaluate the realism of the synthesized interactions through a user study.
  arXiv  Detail & Related papers  (2024-04-19T17:41:05Z)
• ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation [75.0278287071591]
  ThreeDWorld (TDW) is a platform for interactive multi-modal physical simulation. TDW enables simulation of high-fidelity sensory data and physical interactions between mobile agents and objects in rich 3D environments. We present initial experiments enabled by TDW in emerging research directions in computer vision, machine learning, and cognitive science.
  arXiv  Detail & Related papers  (2020-07-09T17:33:27Z)
• Occlusion resistant learning of intuitive physics from videos [52.25308231683798]
  Key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences.
  arXiv  Detail & Related papers  (2020-04-30T19:35:54Z)
• Visual Grounding of Learned Physical Models [66.04898704928517]
  Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors. Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
  arXiv  Detail & Related papers  (2020-04-28T17:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.