CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations

• URL: http://arxiv.org/abs/2107.08994v1
• Date: Mon, 19 Jul 2021 16:13:18 GMT
• Title: CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations
• Authors: Hidenobu Matsuki, Raluca Scona, Jan Czarnowski and Andrew J. Davison
• Abstract summary: State-of-the-art sparse visual SLAM systems provide accurate estimates of the camera trajectory and locations of landmarks. While these sparse maps are useful for localization, they cannot be used for other tasks such as obstacle avoidance or scene understanding. We propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses and sparse points produced by the SLAM system and predicts a dense depth image for every.
• Score: 20.79223452551813
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel dense mapping framework for sparse visual SLAM systems which leverages a compact scene representation. State-of-the-art sparse visual SLAM systems provide accurate and reliable estimates of the camera trajectory and locations of landmarks. While these sparse maps are useful for localization, they cannot be used for other tasks such as obstacle avoidance or scene understanding. In this paper we propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses, keyframes and sparse points produced by the SLAM system and predicts a dense depth image for every keyframe. We build on CodeSLAM and use a variational autoencoder (VAE) which is conditioned on intensity, sparse depth and reprojection error images from sparse SLAM to predict an uncertainty-aware dense depth map. The use of a VAE then enables us to refine the dense depth images through multi-view optimization which improves the consistency of overlapping frames. Our mapper runs in a separate thread in parallel to the SLAM system in a loosely coupled manner. This flexible design allows for integration with arbitrary metric sparse SLAM systems without delaying the main SLAM process. Our dense mapper can be used not only for local mapping but also globally consistent dense 3D reconstruction through TSDF fusion. We demonstrate our system running with ORB-SLAM3 and show accurate dense depth estimation which could enable applications such as robotics and augmented reality.

Related papers

• IG-SLAM: Instant Gaussian SLAM [6.228980850646457]
  3D Gaussian Splatting has recently shown promising results as an alternative scene representation in SLAM systems. We present IG-SLAM, a dense RGB-only SLAM system that employs robust Dense-SLAM methods for tracking and combines them with Gaussian Splatting. We demonstrate competitive performance with state-of-the-art RGB-only SLAM systems while achieving faster operation speeds.
  arXiv  Detail & Related papers  (2024-08-02T09:07:31Z)
• MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements [59.70107451308687]
  We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior rendering by enabling faster scale awareness, and improved trajectory tracking. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit.
  arXiv  Detail & Related papers  (2024-04-01T04:57:41Z)
• Loopy-SLAM: Dense Neural SLAM with Loop Closures [53.11936461015725]
  We introduce Loopy-SLAM that globally optimize poses and the dense 3D model. We use frame-to-model tracking using a data-driven point-based submap generation method and trigger loop closures online by performing global place recognition. Evaluation on the synthetic Replica and real-world TUM-RGBD and ScanNet datasets demonstrate competitive or superior performance in tracking, mapping, and rendering accuracy when compared to existing dense neural RGBD SLAM methods.
  arXiv  Detail & Related papers  (2024-02-14T18:18:32Z)
• Depth Completion with Multiple Balanced Bases and Confidence for Dense Monocular SLAM [34.78726455243436]
  We propose a novel method that integrates a light-weight depth completion network into a sparse SLAM system. Specifically, we present a specifically optimized multi-basis depth completion network, called BBC-Net. BBC-Net can predict multiple balanced bases and a confidence map from a monocular image with sparse points generated by off-the-shelf keypoint-based SLAM systems.
  arXiv  Detail & Related papers  (2023-09-08T06:15:27Z)
• Neural Implicit Dense Semantic SLAM [83.04331351572277]
  We propose a novel RGBD vSLAM algorithm that learns a memory-efficient, dense 3D geometry, and semantic segmentation of an indoor scene in an online manner. Our pipeline combines classical 3D vision-based tracking and loop closing with neural fields-based mapping. Our proposed algorithm can greatly enhance scene perception and assist with a range of robot control problems.
  arXiv  Detail & Related papers  (2023-04-27T23:03:52Z)
• NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM [111.83168930989503]
  NICER-SLAM is a dense RGB SLAM system that simultaneously optimize for camera poses and a hierarchical neural implicit map representation. We show strong performance in dense mapping, tracking, and novel view synthesis, even competitive with recent RGB-D SLAM systems.
  arXiv  Detail & Related papers  (2023-02-07T17:06:34Z)
• ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields [2.0625936401496237]
  ESLAM reads RGB-D frames with unknown camera poses in a sequential manner and incrementally reconstructs the scene representation. ESLAM improves the accuracy of 3D reconstruction and camera localization of state-of-the-art dense visual SLAM methods by more than 50%.
  arXiv  Detail & Related papers  (2022-11-21T18:25:14Z)
• DeepFusion: Real-Time Dense 3D Reconstruction for Monocular SLAM using Single-View Depth and Gradient Predictions [22.243043857097582]
  DeepFusion is capable of producing real-time dense reconstructions on a GPU. It fuses the output of a semi-dense multiview stereo algorithm with the depth and predictions of a CNN in a probabilistic fashion. Based on its performance on synthetic and real-world datasets, we demonstrate that DeepFusion is capable of performing at least as well as other comparable systems.
  arXiv  Detail & Related papers  (2022-07-25T14:55:26Z)
• Simultaneous Localisation and Mapping with Quadric Surfaces [19.516688657045613]
  We introduce a minimal representation for quadric surfaces and show how this can be included in a least-squares formulation. We also show how our representation can be easily extended to include additional constraints on quadrics such as those found in quadrics of revolution.
  arXiv  Detail & Related papers  (2022-03-15T16:26:11Z)
• NICE-SLAM: Neural Implicit Scalable Encoding for SLAM [112.6093688226293]
  NICE-SLAM is a dense SLAM system that incorporates multi-level local information by introducing a hierarchical scene representation. Compared to recent neural implicit SLAM systems, our approach is more scalable, efficient, and robust.
  arXiv  Detail & Related papers  (2021-12-22T18:45:44Z)
• TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo [55.30992853477754]
  We present TANDEM, a real-time monocular tracking and dense framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of alignments. TANDEM shows state-of-the-art real-time 3D reconstruction performance.
  arXiv  Detail & Related papers  (2021-11-14T19:01:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.