Gaussian Splatting SLAM

• URL: http://arxiv.org/abs/2312.06741v2
• Date: Sun, 14 Apr 2024 22:33:27 GMT
• Title: Gaussian Splatting SLAM
• Authors: Hidenobu Matsuki, Riku Murai, Paul H. J. Kelly, Andrew J. Davison,
• Abstract summary: We present the first application of 3D Gaussian Splatting in monocular SLAM. Our method runs live at 3fps, unifying the required representation for accurate tracking, mapping, and high-quality rendering. Several innovations are required to continuously reconstruct 3D scenes with high fidelity from a live camera.
• Score: 16.3858380078553
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present the first application of 3D Gaussian Splatting in monocular SLAM, the most fundamental but the hardest setup for Visual SLAM. Our method, which runs live at 3fps, utilises Gaussians as the only 3D representation, unifying the required representation for accurate, efficient tracking, mapping, and high-quality rendering. Designed for challenging monocular settings, our approach is seamlessly extendable to RGB-D SLAM when an external depth sensor is available. Several innovations are required to continuously reconstruct 3D scenes with high fidelity from a live camera. First, to move beyond the original 3DGS algorithm, which requires accurate poses from an offline Structure from Motion (SfM) system, we formulate camera tracking for 3DGS using direct optimisation against the 3D Gaussians, and show that this enables fast and robust tracking with a wide basin of convergence. Second, by utilising the explicit nature of the Gaussians, we introduce geometric verification and regularisation to handle the ambiguities occurring in incremental 3D dense reconstruction. Finally, we introduce a full SLAM system which not only achieves state-of-the-art results in novel view synthesis and trajectory estimation but also reconstruction of tiny and even transparent objects.

Related papers

• RP-SLAM: Real-time Photorealistic SLAM with Efficient 3D Gaussian Splatting [22.76955981251234]
  RP-SLAM is a 3D Gaussian splatting-based vision SLAM method for monocular and RGB-D cameras. It achieves state-of-the-art map rendering accuracy while ensuring real-time performance and model compactness.
  arXiv  Detail & Related papers  (2024-12-13T05:27:35Z)
• HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction [38.47566815670662]
  HI-SLAM2 is a geometry-aware Gaussian SLAM system that achieves fast and accurate monocular scene reconstruction using only RGB input. We demonstrate significant improvements over existing Neural SLAM methods and even surpass RGB-D-based methods in both reconstruction and rendering quality.
  arXiv  Detail & Related papers  (2024-11-27T01:39:21Z)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting [54.7468067660037]
  PF3plat sets a new state-of-the-art across all benchmarks, supported by comprehensive ablation studies validating our design choices. Our framework capitalizes on fast speed, scalability, and high-quality 3D reconstruction and view synthesis capabilities of 3DGS.
  arXiv  Detail & Related papers  (2024-10-29T15:28:15Z)
• Visual SLAM with 3D Gaussian Primitives and Depth Priors Enabling Novel View Synthesis [11.236094544193605]
  Conventional geometry-based SLAM systems lack dense 3D reconstruction capabilities. We propose a real-time RGB-D SLAM system that incorporates a novel view synthesis technique, 3D Gaussian Splatting.
  arXiv  Detail & Related papers  (2024-08-10T21:23:08Z)
• Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes [50.92217884840301]
  Gaussian Opacity Fields (GOF) is a novel approach for efficient, high-quality, and adaptive surface reconstruction in scenes. GOF is derived from ray-tracing-based volume rendering of 3D Gaussians. GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis.
  arXiv  Detail & Related papers  (2024-04-16T17:57:19Z)
• MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements [59.70107451308687]
  We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior rendering by enabling faster scale awareness, and improved trajectory tracking. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit.
  arXiv  Detail & Related papers  (2024-04-01T04:57:41Z)
• 2D Gaussian Splatting for Geometrically Accurate Radiance Fields [50.056790168812114]
  3D Gaussian Splatting (3DGS) has recently revolutionized radiance field reconstruction, achieving high quality novel view synthesis and fast rendering speed without baking. We present 2D Gaussian Splatting (2DGS), a novel approach to model and reconstruct geometrically accurate radiance fields from multi-view images. We demonstrate that our differentiable terms allows for noise-free and detailed geometry reconstruction while maintaining competitive appearance quality, fast training speed, and real-time rendering.
  arXiv  Detail & Related papers  (2024-03-26T17:21:24Z)
• NEDS-SLAM: A Neural Explicit Dense Semantic SLAM Framework using 3D Gaussian Splatting [5.655341825527482]
  NEDS-SLAM is a dense semantic SLAM system based on 3D Gaussian representation. We propose a Spatially Consistent Feature Fusion model to reduce the effect of erroneous estimates from pre-trained segmentation head. We employ a lightweight encoder-decoder to compress the high-dimensional semantic features into a compact 3D Gaussian representation.
  arXiv  Detail & Related papers  (2024-03-18T11:31:03Z)
• GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting [51.96353586773191]
  We introduce textbfGS-SLAM that first utilizes 3D Gaussian representation in the Simultaneous Localization and Mapping system. Our method utilizes a real-time differentiable splatting rendering pipeline that offers significant speedup to map optimization and RGB-D rendering. Our method achieves competitive performance compared with existing state-of-the-art real-time methods on the Replica, TUM-RGBD datasets.
  arXiv  Detail & Related papers  (2023-11-20T12:08:23Z)
• NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM [111.83168930989503]
  NICER-SLAM is a dense RGB SLAM system that simultaneously optimize for camera poses and a hierarchical neural implicit map representation. We show strong performance in dense mapping, tracking, and novel view synthesis, even competitive with recent RGB-D SLAM systems.
  arXiv  Detail & Related papers  (2023-02-07T17:06:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.