Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic Reconstruction

• URL: http://arxiv.org/abs/2411.06602v1
• Date: Sun, 10 Nov 2024 21:30:16 GMT
• Title: Adaptive and Temporally Consistent Gaussian Surfels for Multi-view Dynamic Reconstruction
• Authors: Decai Chen, Brianne Oberson, Ingo Feldmann, Oliver Schreer, Anna Hilsmann, Peter Eisert,
• Abstract summary: AT-GS is a novel method for reconstructing high-quality dynamic surfaces from multi-view videos through per-frame incremental optimization. We reduce temporal jittering in dynamic surfaces by ensuring consistency in curvature maps across consecutive frames. Our method achieves superior accuracy and temporal coherence in dynamic surface reconstruction, delivering high-fidelity space-time novel view synthesis.
• Score: 3.9363268745580426
• License:
• Abstract: 3D Gaussian Splatting has recently achieved notable success in novel view synthesis for dynamic scenes and geometry reconstruction in static scenes. Building on these advancements, early methods have been developed for dynamic surface reconstruction by globally optimizing entire sequences. However, reconstructing dynamic scenes with significant topology changes, emerging or disappearing objects, and rapid movements remains a substantial challenge, particularly for long sequences. To address these issues, we propose AT-GS, a novel method for reconstructing high-quality dynamic surfaces from multi-view videos through per-frame incremental optimization. To avoid local minima across frames, we introduce a unified and adaptive gradient-aware densification strategy that integrates the strengths of conventional cloning and splitting techniques. Additionally, we reduce temporal jittering in dynamic surfaces by ensuring consistency in curvature maps across consecutive frames. Our method achieves superior accuracy and temporal coherence in dynamic surface reconstruction, delivering high-fidelity space-time novel view synthesis, even in complex and challenging scenes. Extensive experiments on diverse multi-view video datasets demonstrate the effectiveness of our approach, showing clear advantages over baseline methods. Project page: \url{https://fraunhoferhhi.github.io/AT-GS}

Related papers

• 4D Gaussian Splatting with Scale-aware Residual Field and Adaptive Optimization for Real-time Rendering of Temporally Complex Dynamic Scenes [19.24815625343669]
  SaRO-GS is a novel dynamic scene representation capable of achieving real-time rendering. To handle temporally complex dynamic scenes, we introduce a Scale-aware Residual Field. Our method has demonstrated state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-12-09T08:44:19Z)
• Urban4D: Semantic-Guided 4D Gaussian Splatting for Urban Scene Reconstruction [86.4386398262018]
  Urban4D is a semantic-guided decomposition strategy inspired by advances in deep 2D semantic map generation. Our approach distinguishes potentially dynamic objects through reliable semantic Gaussians. Experiments on real-world datasets demonstrate that Urban4D achieves comparable or better quality than previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-12-04T16:59:49Z)
• DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenes [71.61083731844282]
  We present DeSiRe-GS, a self-supervised gaussian splatting representation. It enables effective static-dynamic decomposition and high-fidelity surface reconstruction in complex driving scenarios.
  arXiv  Detail & Related papers  (2024-11-18T05:49:16Z)
• ReMatching Dynamic Reconstruction Flow [55.272357926111454]
  We introduce the ReMatching framework, designed to improve generalization quality by incorporating deformation priors into dynamic reconstruction models. The framework is highly adaptable and can be applied to various dynamic representations. Our evaluations on popular benchmarks involving both synthetic and real-world dynamic scenes demonstrate a clear improvement in reconstruction accuracy of current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-11-01T16:09:33Z)
• SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstruction [24.33543853742041]
  3D Gaussian Splatting (3DGS) has emerged as a practical and scalable reconstruction method. We propose an optimization strategy that effectively regularizes splat features by modeling them as the outputs of a corresponding implicit neural field. Our approach effectively handles static and dynamic cases, as demonstrated by extensive testing across different setups and scene complexities.
  arXiv  Detail & Related papers  (2024-09-17T14:04:20Z)
• SMORE: Simulataneous Map and Object REconstruction [66.66729715211642]
  We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR. We take a holistic perspective and optimize a compositional model of a dynamic scene that decomposes the world into rigidly-moving objects and the background.
  arXiv  Detail & Related papers  (2024-06-19T23:53:31Z)
• Modeling Ambient Scene Dynamics for Free-view Synthesis [31.233859111566613]
  We introduce a novel method for dynamic free-view synthesis of an ambient scenes from a monocular capture. Our method builds upon the recent advancements in 3D Gaussian Splatting (3DGS) that can faithfully reconstruct complex static scenes.
  arXiv  Detail & Related papers  (2024-06-13T17:59:11Z)
• SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes [59.23385953161328]
  Novel view synthesis for dynamic scenes is still a challenging problem in computer vision and graphics. We propose a new representation that explicitly decomposes the motion and appearance of dynamic scenes into sparse control points and dense Gaussians. Our method can enable user-controlled motion editing while retaining high-fidelity appearances.
  arXiv  Detail & Related papers  (2023-12-04T11:57:14Z)
• Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Rendering [36.111845416439095]
  We present a unified representation model, called Periodic Vibration Gaussian (PVG) PVG builds upon the efficient 3D Gaussian splatting technique, originally designed for static scene representation. PVG exhibits 900-fold acceleration in rendering over the best alternative.
  arXiv  Detail & Related papers  (2023-11-30T13:53:50Z)
• NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction [95.37644907940857]
  We propose a fast neural surface reconstruction approach, called NeuS2. NeuS2 achieves two orders of magnitude improvement in terms of acceleration without compromising reconstruction quality. We extend our method for fast training of dynamic scenes, with a proposed incremental training strategy and a novel global transformation prediction component.
  arXiv  Detail & Related papers  (2022-12-10T07:19:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.