Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians

• URL: http://arxiv.org/abs/2403.09434v3
• Date: Fri, 19 Jul 2024 10:23:21 GMT
• Title: Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians
• Authors: Licheng Zhong, Hong-Xing Yu, Jiajun Wu, Yunzhu Li,
• Abstract summary: Spring-Gaus is a 3D physical object representation for reconstructing and simulating elastic objects from videos of the object from multiple viewpoints. We develop and integrate a 3D Spring-Mass model into 3D Gaussian kernels, enabling the reconstruction of the visual appearance, shape, and physical dynamics of the object. We evaluate Spring-Gaus on both synthetic and real-world datasets, demonstrating accurate reconstruction and simulation of elastic objects.
• Score: 23.572267290979045
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing and simulating elastic objects from visual observations is crucial for applications in computer vision and robotics. Existing methods, such as 3D Gaussians, model 3D appearance and geometry, but lack the ability to estimate physical properties for objects and simulate them. The core challenge lies in integrating an expressive yet efficient physical dynamics model. We propose Spring-Gaus, a 3D physical object representation for reconstructing and simulating elastic objects from videos of the object from multiple viewpoints. In particular, we develop and integrate a 3D Spring-Mass model into 3D Gaussian kernels, enabling the reconstruction of the visual appearance, shape, and physical dynamics of the object. Our approach enables future prediction and simulation under various initial states and environmental properties. We evaluate Spring-Gaus on both synthetic and real-world datasets, demonstrating accurate reconstruction and simulation of elastic objects. Project page: https://zlicheng.com/spring_gaus/.

Related papers

• Automated 3D Physical Simulation of Open-world Scene with Gaussian Splatting [22.40115216094332]
  We present Sim Anything, a physics-based approach that endows static 3D objects with interactive dynamics. Inspired by human visual reasoning, we propose MLLM-based Physical Property Perception. We also simulate objects in an open-world scene with particles sampled via the Physical-Geometric Adaptive Sampling.
  arXiv  Detail & Related papers  (2024-11-19T12:52:21Z)
• GASP: Gaussian Splatting for Physic-Based Simulations [0.42881773214459123]
  Existing physics models use additional meshing mechanisms, including triangle or tetrahedron meshing, marching cubes, or cage meshes. We modify the physics grounded Newtonian dynamics to align with 3D Gaussian components. Resulting solution can be integrated into any physics engine that can be treated as a black box.
  arXiv  Detail & Related papers  (2024-09-09T17:28:57Z)
• GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulation [60.33467489955188]
  This paper studies the problem of estimating physical properties (system identification) through visual observations. To facilitate geometry-aware guidance in physical property estimation, we introduce a novel hybrid framework. We propose a new dynamic 3D Gaussian framework based on motion factorization to recover the object as 3D Gaussian point sets. In addition to the extracted object surfaces, the Gaussian-informed continuum also enables the rendering of object masks during simulations.
  arXiv  Detail & Related papers  (2024-06-21T07:37:17Z)
• Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video [58.043569985784806]
  We introduce latent intuitive physics, a transfer learning framework for physics simulation. It can infer hidden properties of fluids from a single 3D video and simulate the observed fluid in novel scenes. We validate our model in three ways: (i) novel scene simulation with the learned visual-world physics, (ii) future prediction of the observed fluid dynamics, and (iii) supervised particle simulation.
  arXiv  Detail & Related papers  (2024-06-18T16:37:44Z)
• Physics3D: Learning Physical Properties of 3D Gaussians via Video Diffusion [35.71595369663293]
  We propose textbfPhysics3D, a novel method for learning various physical properties of 3D objects through a video diffusion model. Our approach involves designing a highly generalizable physical simulation system based on a viscoelastic material model. Experiments demonstrate the effectiveness of our method with both elastic and plastic materials.
  arXiv  Detail & Related papers  (2024-06-06T17:59:47Z)
• DreamPhysics: Learning Physical Properties of Dynamic 3D Gaussians with Video Diffusion Priors [75.83647027123119]
  We propose to learn the physical properties of a material field with video diffusion priors. We then utilize a physics-based Material-Point-Method simulator to generate 4D content with realistic motions.
  arXiv  Detail & Related papers  (2024-06-03T16:05:25Z)
• PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generation [62.53760963292465]
  PhysDreamer is a physics-based approach that endows static 3D objects with interactive dynamics. We present our approach on diverse examples of elastic objects and evaluate the realism of the synthesized interactions through a user study.
  arXiv  Detail & Related papers  (2024-04-19T17:41:05Z)
• Feature Splatting: Language-Driven Physics-Based Scene Synthesis and Editing [11.46530458561589]
  We introduce Feature Splatting, an approach that unifies physics-based dynamic scene synthesis with rich semantics. Our first contribution is a way to distill high-quality, object-centric vision-language features into 3D Gaussians. Our second contribution is a way to synthesize physics-based dynamics from an otherwise static scene using a particle-based simulator.
  arXiv  Detail & Related papers  (2024-04-01T16:31:04Z)
• Occlusion resistant learning of intuitive physics from videos [52.25308231683798]
  Key ability for artificial systems is to understand physical interactions between objects, and predict future outcomes of a situation. This ability, often referred to as intuitive physics, has recently received attention and several methods were proposed to learn these physical rules from video sequences.
  arXiv  Detail & Related papers  (2020-04-30T19:35:54Z)
• Predicting the Physical Dynamics of Unseen 3D Objects [65.49291702488436]
  We focus on predicting the dynamics of 3D objects on a plane that have just been subjected to an impulsive force. Our approach can generalize to object shapes and initial conditions that were unseen during training. Our model can support training with data from both a physics engine or the real world.
  arXiv  Detail & Related papers  (2020-01-16T06:27:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.