PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics

• URL: http://arxiv.org/abs/2311.12198v3
• Date: Mon, 15 Apr 2024 06:04:55 GMT
• Title: PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics
• Authors: Tianyi Xie, Zeshun Zong, Yuxing Qiu, Xuan Li, Yutao Feng, Yin Yang, Chenfanfu Jiang,
• Abstract summary: We introduce PhysGaussian, a new method that seamlessly integrates physically grounded Newtonian dynamics within 3D Gaussians. A defining characteristic of our method is the seamless integration between physical simulation and visual rendering. Our method demonstrates exceptional versatility across a wide variety of materials.
• Score: 22.4647573375673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce PhysGaussian, a new method that seamlessly integrates physically grounded Newtonian dynamics within 3D Gaussians to achieve high-quality novel motion synthesis. Employing a custom Material Point Method (MPM), our approach enriches 3D Gaussian kernels with physically meaningful kinematic deformation and mechanical stress attributes, all evolved in line with continuum mechanics principles. A defining characteristic of our method is the seamless integration between physical simulation and visual rendering: both components utilize the same 3D Gaussian kernels as their discrete representations. This negates the necessity for triangle/tetrahedron meshing, marching cubes, "cage meshes," or any other geometry embedding, highlighting the principle of "what you see is what you simulate (WS$^2$)." Our method demonstrates exceptional versatility across a wide variety of materials--including elastic entities, metals, non-Newtonian fluids, and granular materials--showcasing its strong capabilities in creating diverse visual content with novel viewpoints and movements. Our project page is at: https://xpandora.github.io/PhysGaussian/

Related papers

• GASP: Gaussian Splatting for Physic-Based Simulations [0.42881773214459123]
  Existing physics models use additional meshing mechanisms, including triangle or tetrahedron meshing, marching cubes, or cage meshes. We modify the physics grounded Newtonian dynamics to align with 3D Gaussian components. Resulting solution can be integrated into any physics engine that can be treated as a black box.
  arXiv  Detail & Related papers  (2024-09-09T17:28:57Z)
• Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos [58.22272760132996]
  We show that existing 4D Gaussian methods dramatically fail in this setup because the monocular setting is underconstrained. We propose Dynamic Gaussian Marbles, which consist of three core modifications that target the difficulties of the monocular setting. We evaluate on the Nvidia Dynamic Scenes dataset and the DyCheck iPhone dataset, and show that Gaussian Marbles significantly outperforms other Gaussian baselines in quality.
  arXiv  Detail & Related papers  (2024-06-26T19:37:07Z)
• GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation [60.33467489955188]
  This paper studies the problem of estimating physical properties (system identification) through visual observations. To facilitate geometry-aware guidance in physical property estimation, we introduce a novel hybrid framework. We propose a new dynamic 3D Gaussian framework based on motion factorization to recover the object as 3D Gaussian point sets. In addition to the extracted object surfaces, the Gaussian-informed continuum also enables the rendering of object masks during simulations.
  arXiv  Detail & Related papers  (2024-06-21T07:37:17Z)
• Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
  We introduce latent intuitive physics, a transfer learning framework for physics simulation. It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes. We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
  arXiv  Detail & Related papers  (2024-06-18T16:37:44Z)
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion [35.71595369663293]
  We propose textbfPhysics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model. Experiments demonstrate the effectiveness of our method with both elastic and plastic materials.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• DreamPhysics: Learning Physical Properties of Dynamic 3D Gaussians with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors. We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)
• Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians [23.572267290979045]
  Spring-Gaus is a 3D physical object representation for reconstructing and simulating elastic objects from videos of the object from multiple viewpoints. We develop and integrate a 3D Spring-Mass model into 3D Gaussian kernels, enabling the reconstruction of the visual appearance, shape, and physical dynamics of the object. We evaluate Spring-Gaus on both synthetic and real-world datasets, demonstrating accurate reconstruction and simulation of elastic objects.
  arXiv  Detail & Related papers  (2024-03-14T14:25:10Z)
• Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering [41.589093951039814]
  We integrate physics-based animations of solids and fluids with 3D Gaussian Splatting (3DGS) to create novel effects in virtual scenes reconstructed using 3DGS. Our framework is capable of realistically reproducing surface highlights on dynamic fluids and facilitating interactions between scene objects and fluids from new views.
  arXiv  Detail & Related papers  (2024-01-27T06:45:22Z)
• Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis [58.5779956899918]
  We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.
  arXiv  Detail & Related papers  (2023-08-18T17:59:21Z)
• PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification [64.61198351207752]
  Existing approaches to system identification (estimating the physical parameters of an object) from videos assume known object geometries. In this work, we aim to identify parameters characterizing a physical system from a set of multi-view videos without any assumption on object geometry or topology. We propose "Physics Augmented Continuum Neural Radiance Fields" (PAC-NeRF), to estimate both the unknown geometry and physical parameters of highly dynamic objects from multi-view videos.
  arXiv  Detail & Related papers  (2023-03-09T18:59:50Z)
• PhysGraph: Physics-Based Integration Using Graph Neural Networks [9.016253794897874]
  We focus on the detail enhancement of coarse clothing geometry which has many applications including computer games, virtual reality and virtual try-on. Our contribution is based on a simple observation: evaluating forces is computationally relatively cheap for traditional simulation methods. We demonstrate that this idea leads to a learnable module that can be trained on basic internal forces of small mesh patches.
  arXiv  Detail & Related papers  (2023-01-27T16:47:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.