DENSER: 3D Gaussians Splatting for Scene Reconstruction of Dynamic Urban Environments

• URL: http://arxiv.org/abs/2409.10041v1
• Date: Mon, 16 Sep 2024 07:11:58 GMT
• Title: DENSER: 3D Gaussians Splatting for Scene Reconstruction of Dynamic Urban Environments
• Authors: Mahmud A. Mohamad, Gamal Elghazaly, Arthur Hubert, Raphael Frank,
• Abstract summary: We propose DENSER, a framework that significantly enhances the representation of dynamic objects. The proposed approach significantly outperforms state-of-the-art methods by a wide margin.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents DENSER, an efficient and effective approach leveraging 3D Gaussian splatting (3DGS) for the reconstruction of dynamic urban environments. While several methods for photorealistic scene representations, both implicitly using neural radiance fields (NeRF) and explicitly using 3DGS have shown promising results in scene reconstruction of relatively complex dynamic scenes, modeling the dynamic appearance of foreground objects tend to be challenging, limiting the applicability of these methods to capture subtleties and details of the scenes, especially far dynamic objects. To this end, we propose DENSER, a framework that significantly enhances the representation of dynamic objects and accurately models the appearance of dynamic objects in the driving scene. Instead of directly using Spherical Harmonics (SH) to model the appearance of dynamic objects, we introduce and integrate a new method aiming at dynamically estimating SH bases using wavelets, resulting in better representation of dynamic objects appearance in both space and time. Besides object appearance, DENSER enhances object shape representation through densification of its point cloud across multiple scene frames, resulting in faster convergence of model training. Extensive evaluations on KITTI dataset show that the proposed approach significantly outperforms state-of-the-art methods by a wide margin. Source codes and models will be uploaded to this repository https://github.com/sntubix/denser

Related papers

• Template-free Articulated Gaussian Splatting for Real-time Reposable Dynamic View Synthesis [21.444265403717015]
  We propose a novel approach to automatically discover the associated skeleton model for dynamic objects from videos without the need for object-specific templates. Treating superpoints as rigid parts, we can discover the underlying skeleton model through intuitive cues and optimize it using the kinematic model. Experiments demonstrate the effectiveness and efficiency of our method in obtaining re-posable 3D objects.
  arXiv  Detail & Related papers  (2024-12-07T07:35:09Z)
• Urban4D: Semantic-Guided 4D Gaussian Splatting for Urban Scene Reconstruction [86.4386398262018]
  Urban4D is a semantic-guided decomposition strategy inspired by advances in deep 2D semantic map generation. Our approach distinguishes potentially dynamic objects through reliable semantic Gaussians. Experiments on real-world datasets demonstrate that Urban4D achieves comparable or better quality than previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-12-04T16:59:49Z)
• MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion [118.74385965694694]
  We present Motion DUSt3R (MonST3R), a novel geometry-first approach that directly estimates per-timestep geometry from dynamic scenes. By simply estimating a pointmap for each timestep, we can effectively adapt DUST3R's representation, previously only used for static scenes, to dynamic scenes. We show that by posing the problem as a fine-tuning task, identifying several suitable datasets, and strategically training the model on this limited data, we can surprisingly enable the model to handle dynamics.
  arXiv  Detail & Related papers  (2024-10-04T18:00:07Z)
• Dynamic Scene Understanding through Object-Centric Voxelization and Neural Rendering [57.895846642868904]
  We present a 3D generative model named DynaVol-S for dynamic scenes that enables object-centric learning. voxelization infers per-object occupancy probabilities at individual spatial locations. Our approach integrates 2D semantic features to create 3D semantic grids, representing the scene through multiple disentangled voxel grids.
  arXiv  Detail & Related papers  (2024-07-30T15:33:58Z)
• Dynamic 3D Gaussian Fields for Urban Areas [60.64840836584623]
  We present an efficient neural 3D scene representation for novel-view synthesis (NVS) in large-scale, dynamic urban areas. We propose 4DGF, a neural scene representation that scales to large-scale dynamic urban areas.
  arXiv  Detail & Related papers  (2024-06-05T12:07:39Z)
• Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Rendering [36.111845416439095]
  We present a unified representation model, called Periodic Vibration Gaussian (PVG) PVG builds upon the efficient 3D Gaussian splatting technique, originally designed for static scene representation. PVG exhibits 900-fold acceleration in rendering over the best alternative.
  arXiv  Detail & Related papers  (2023-11-30T13:53:50Z)
• Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis [58.5779956899918]
  We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.
  arXiv  Detail & Related papers  (2023-08-18T17:59:21Z)
• DOT: Dynamic Object Tracking for Visual SLAM [83.69544718120167]
  DOT combines instance segmentation and multi-view geometry to generate masks for dynamic objects. To determine which objects are actually moving, DOT segments first instances of potentially dynamic objects and then, with the estimated camera motion, tracks such objects by minimizing the photometric reprojection error. Our results show that our approach improves significantly the accuracy and robustness of ORB-SLAM 2, especially in highly dynamic scenes.
  arXiv  Detail & Related papers  (2020-09-30T18:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.