Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering

• URL: http://arxiv.org/abs/2401.15318v2
• Date: Tue, 23 Jul 2024 04:05:53 GMT
• Title: Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering
• Authors: Yutao Feng, Xiang Feng, Yintong Shang, Ying Jiang, Chang Yu, Zeshun Zong, Tianjia Shao, Hongzhi Wu, Kun Zhou, Chenfanfu Jiang, Yin Yang,
• Abstract summary: We integrate physics-based animations of solids and fluids with 3D Gaussian Splatting (3DGS) to create novel effects in virtual scenes reconstructed using 3DGS. Our framework is capable of realistically reproducing surface highlights on dynamic fluids and facilitating interactions between scene objects and fluids from new views.
• Score: 41.589093951039814
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We demonstrate the feasibility of integrating physics-based animations of solids and fluids with 3D Gaussian Splatting (3DGS) to create novel effects in virtual scenes reconstructed using 3DGS. Leveraging the coherence of the Gaussian Splatting and Position-Based Dynamics (PBD) in the underlying representation, we manage rendering, view synthesis, and the dynamics of solids and fluids in a cohesive manner. Similar to GaussianShader, we enhance each Gaussian kernel with an added normal, aligning the kernel's orientation with the surface normal to refine the PBD simulation. This approach effectively eliminates spiky noises that arise from rotational deformation in solids. It also allows us to integrate physically based rendering to augment the dynamic surface reflections on fluids. Consequently, our framework is capable of realistically reproducing surface highlights on dynamic fluids and facilitating interactions between scene objects and fluids from new views. For more information, please visit our project page at \url{https://gaussiansplashing.github.io/}.

Related papers

• BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splatting [10.918133974256913]
  We present Bidirectional Gaussian Primitives, an image-based novel view synthesis technique. Our approach integrates light intrinsic decomposition into the Gaussian splatting framework, enabling real-time relighting of 3D objects.
  arXiv  Detail & Related papers  (2024-08-23T21:04:40Z)
• Motion-aware 3D Gaussian Splatting for Efficient Dynamic Scene Reconstruction [89.53963284958037]
  We propose a novel motion-aware enhancement framework for dynamic scene reconstruction. Specifically, we first establish a correspondence between 3D Gaussian movements and pixel-level flow. For the prevalent deformation-based paradigm that presents a harder optimization problem, a transient-aware deformation auxiliary module is proposed.
  arXiv  Detail & Related papers  (2024-03-18T03:46:26Z)
• GaussianStyle: Gaussian Head Avatar via StyleGAN [64.85782838199427]
  We propose a novel framework that integrates the volumetric strengths of 3DGS with the powerful implicit representation of StyleGAN. We show that our method achieves state-of-the-art performance in reenactment, novel view synthesis, and animation.
  arXiv  Detail & Related papers  (2024-02-01T18:14:42Z)
• GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesis [17.572987038801475]
  We propose a method for dynamic scene reconstruction using deformable 3D Gaussians. The differentiable pipeline is optimized end-to-end with a self-supervised rendering. Our method results are comparable to state-of-the-art neural radiance field methods.
  arXiv  Detail & Related papers  (2023-12-18T18:59:03Z)
• DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes [57.12439406121721]
  We present DrivingGaussian, an efficient and effective framework for surrounding dynamic autonomous driving scenes. For complex scenes with moving objects, we first sequentially and progressively model the static background of the entire scene. We then leverage a composite dynamic Gaussian graph to handle multiple moving objects. We further use a LiDAR prior for Gaussian Splatting to reconstruct scenes with greater details and maintain panoramic consistency.
  arXiv  Detail & Related papers  (2023-12-13T06:30:51Z)
• SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes [59.23385953161328]
  Novel view synthesis for dynamic scenes is still a challenging problem in computer vision and graphics. We propose a new representation that explicitly decomposes the motion and appearance of dynamic scenes into sparse control points and dense Gaussians. Our method can enable user-controlled motion editing while retaining high-fidelity appearances.
  arXiv  Detail & Related papers  (2023-12-04T11:57:14Z)
• PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics [22.4647573375673]
  We introduce PhysGaussian, a new method that seamlessly integrates physically grounded Newtonian dynamics within 3D Gaussians. A defining characteristic of our method is the seamless integration between physical simulation and visual rendering. Our method demonstrates exceptional versatility across a wide variety of materials.
  arXiv  Detail & Related papers  (2023-11-20T21:34:52Z)
• Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis [58.5779956899918]
  We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.
  arXiv  Detail & Related papers  (2023-08-18T17:59:21Z)
• Inferring Fluid Dynamics via Inverse Rendering [37.87293082992423]
  Humans have a strong intuitive understanding of physical processes such as fluid falling by just a glimpse of such a scene picture. This work achieves such a photo-to-fluid reconstruction functionality learned from unannotated videos.
  arXiv  Detail & Related papers  (2023-04-10T08:23:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.