Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic Reconstruction

• URL: http://arxiv.org/abs/2411.06602v1
• Date: Sun, 10 Nov 2024 21:30:16 GMT
• Title: Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic Reconstruction
• Authors: Decai Chen, Brianne Oberson, Ingo Feldmann, Oliver Schreer, Anna Hilsmann, Peter Eisert,
• Abstract summary: AT-GS is a novel method for reconstructing high-quality dynamic surfaces from multi-view videos through per-frame incremental optimization. We reduce temporal jittering in dynamic surfaces by ensuring consistency in curvature maps across consecutive frames. Our method achieves superior accuracy and temporal coherence in dynamic surface reconstruction, delivering high-fidelity space-time novel view synthesis.
• Score: 3.9363268745580426
• License:
• Abstract: 3D Gaussian Splatting has recently achieved notable success in novel view synthesis for dynamic scenes and geometry reconstruction in static scenes. Building on these advancements, early methods have been developed for dynamic surface reconstruction by globally optimizing entire sequences. However, reconstructing dynamic scenes with significant topology changes, emerging or disappearing objects, and rapid movements remains a substantial challenge, particularly for long sequences. To address these issues, we propose AT-GS, a novel method for reconstructing high-quality dynamic surfaces from multi-view videos through per-frame incremental optimization. To avoid local minima across frames, we introduce a unified and adaptive gradient-aware densification strategy that integrates the strengths of conventional cloning and splitting techniques. Additionally, we reduce temporal jittering in dynamic surfaces by ensuring consistency in curvature maps across consecutive frames. Our method achieves superior accuracy and temporal coherence in dynamic surface reconstruction, delivering high-fidelity space-time novel view synthesis, even in complex and challenging scenes. Extensive experiments on diverse multi-view video datasets demonstrate the effectiveness of our approach, showing clear advantages over baseline methods. Project page: \url{https://fraunhoferhhi.github.io/AT-GS}

Related papers

• DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes [71.61083731844282]
  We present DeSiRe-GS, a self-supervised gaussian splatting representation. It enables effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios.
  arXiv  Detail & Related papers  (2024-11-18T05:49:16Z)
• ReMatching Dynamic Reconstruction Flow [55.272357926111454]
  We introduce the ReMatching framework, designed to improve generalization quality by incorporating deformation priors into dynamic reconstruction models. The framework is highly adaptable and can be applied to various dynamic representations. Our evaluations on popular benchmarks involving both synthetic and real-world dynamic scenes demonstrate a clear improvement in reconstruction accuracy of current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-11-01T16:09:33Z)
• MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion [118.74385965694694]
  We present Motion DUSt3R (MonST3R), a novel geometry-first approach that directly estimates per-timestep geometry from dynamic scenes. By simply estimating a pointmap for each timestep, we can effectively adapt DUST3R's representation, previously only used for static scenes, to dynamic scenes. We show that by posing the problem as a fine-tuning task, identifying several suitable datasets, and strategically training the model on this limited data, we can surprisingly enable the model to handle dynamics.
  arXiv  Detail & Related papers  (2024-10-04T18:00:07Z)
• SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction [24.33543853742041]
  3D Gaussian Splatting (3DGS) has emerged as a practical and scalable reconstruction method. We propose an optimization strategy that effectively regularizes splat features by modeling them as the outputs of a corresponding implicit neural field. Our approach effectively handles static and dynamic cases, as demonstrated by extensive testing across different setups and scene complexities.
  arXiv  Detail & Related papers  (2024-09-17T14:04:20Z)
• Gaussian Splatting LK [0.11249583407496218]
  This paper investigates the potential of regularizing the native warp field within the dynamic Gaussian Splatting framework. We show that we can exploit knowledge innate to the forward warp field network to derive an analytical velocity field. This derived Lucas-Kanade style analytical regularization enables our method to achieve superior performance in reconstructing highly dynamic scenes.
  arXiv  Detail & Related papers  (2024-07-16T01:50:43Z)
• Simultaneous Map and Object Reconstruction [66.66729715211642]
  We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR. We take inspiration from recent novel view synthesis methods and pose the reconstruction problem as a global optimization. By careful modeling of continuous-time motion, our reconstructions can compensate for the rolling shutter effects of rotating LiDAR sensors.
  arXiv  Detail & Related papers  (2024-06-19T23:53:31Z)
• Modeling Ambient Scene Dynamics for Free-view Synthesis [31.233859111566613]
  We introduce a novel method for dynamic free-view synthesis of an ambient scenes from a monocular capture. Our method builds upon the recent advancements in 3D Gaussian Splatting (3DGS) that can faithfully reconstruct complex static scenes.
  arXiv  Detail & Related papers  (2024-06-13T17:59:11Z)
• SWinGS: Sliding Windows for Dynamic 3D Gaussian Splatting [7.553079256251747]
  We extend 3D Gaussian Splatting to reconstruct dynamic scenes. We produce high-quality renderings of general dynamic scenes with competitive quantitative performance. Our method can be viewed in real-time in our dynamic interactive viewer.
  arXiv  Detail & Related papers  (2023-12-20T03:54:03Z)
• SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes [59.23385953161328]
  Novel view synthesis for dynamic scenes is still a challenging problem in computer vision and graphics. We propose a new representation that explicitly decomposes the motion and appearance of dynamic scenes into sparse control points and dense Gaussians. Our method can enable user-controlled motion editing while retaining high-fidelity appearances.
  arXiv  Detail & Related papers  (2023-12-04T11:57:14Z)
• Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Rendering [36.111845416439095]
  We present a unified representation model, called Periodic Vibration Gaussian (PVG) PVG builds upon the efficient 3D Gaussian splatting technique, originally designed for static scene representation. PVG exhibits 900-fold acceleration in rendering over the best alternative.
  arXiv  Detail & Related papers  (2023-11-30T13:53:50Z)
• NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction [95.37644907940857]
  We propose a fast neural surface reconstruction approach, called NeuS2. NeuS2 achieves two orders of magnitude improvement in terms of acceleration without compromising reconstruction quality. We extend our method for fast training of dynamic scenes, with a proposed incremental training strategy and a novel global transformation prediction component.
  arXiv  Detail & Related papers  (2022-12-10T07:19:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.