Depth Completion with Multiple Balanced Bases and Confidence for Dense Monocular SLAM

• URL: http://arxiv.org/abs/2309.04145v2
• Date: Wed, 20 Sep 2023 07:54:04 GMT
• Title: Depth Completion with Multiple Balanced Bases and Confidence for Dense Monocular SLAM
• Authors: Weijian Xie, Guanyi Chu, Quanhao Qian, Yihao Yu, Hai Li, Danpeng Chen, Shangjin Zhai, Nan Wang, Hujun Bao, Guofeng Zhang
• Abstract summary: We propose a novel method that integrates a light-weight depth completion network into a sparse SLAM system. Specifically, we present a specifically optimized multi-basis depth completion network, called BBC-Net. BBC-Net can predict multiple balanced bases and a confidence map from a monocular image with sparse points generated by off-the-shelf keypoint-based SLAM systems.
• Score: 34.78726455243436
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dense SLAM based on monocular cameras does indeed have immense application value in the field of AR/VR, especially when it is performed on a mobile device. In this paper, we propose a novel method that integrates a light-weight depth completion network into a sparse SLAM system using a multi-basis depth representation, so that dense mapping can be performed online even on a mobile phone. Specifically, we present a specifically optimized multi-basis depth completion network, called BBC-Net, tailored to the characteristics of traditional sparse SLAM systems. BBC-Net can predict multiple balanced bases and a confidence map from a monocular image with sparse points generated by off-the-shelf keypoint-based SLAM systems. The final depth is a linear combination of predicted depth bases that can be optimized by tuning the corresponding weights. To seamlessly incorporate the weights into traditional SLAM optimization and ensure efficiency and robustness, we design a set of depth weight factors, which makes our network a versatile plug-in module, facilitating easy integration into various existing sparse SLAM systems and significantly enhancing global depth consistency through bundle adjustment. To verify the portability of our method, we integrate BBC-Net into two representative SLAM systems. The experimental results on various datasets show that the proposed method achieves better performance in monocular dense mapping than the state-of-the-art methods. We provide an online demo running on a mobile phone, which verifies the efficiency and mapping quality of the proposed method in real-world scenarios.

Related papers

• GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM [53.6402869027093]
  We propose an efficient RGB-only dense SLAM system using a flexible neural point cloud representation scene. We also introduce a novel DSPO layer for bundle adjustment which optimize the pose and depth of implicits along with the scale of the monocular depth.
  arXiv  Detail & Related papers  (2024-03-28T16:32:06Z)
• 360ORB-SLAM: A Visual SLAM System for Panoramic Images with Depth Completion Network [18.23570356507258]
  This paper proposes a 360ORB-SLAM system for panoramic images that combines with a depth completion network. The proposed method achieves superior scale accuracy compared to existing monocular SLAM methods. The integration of the depth completion network enhances system stability and mitigates the impact of dynamic elements on SLAM performance.
  arXiv  Detail & Related papers  (2024-01-19T08:52:24Z)
• HDPV-SLAM: Hybrid Depth-augmented Panoramic Visual SLAM for Mobile Mapping System with Tilted LiDAR and Panoramic Visual Camera [4.2421412410466575]
  This paper proposes a novel visual simultaneous localization and mapping (SLAM) system called Hybrid Depth-augmented Panoramic Visual SLAM (HDPV-SLAM) It employs a panoramic camera and a tilted multi-beam LiDAR scanner to generate accurate and metrically-scaled trajectories. It aims to solve the two major issues hindering the performance of similar SLAM systems.
  arXiv  Detail & Related papers  (2023-01-27T16:25:28Z)
• Structure PLP-SLAM: Efficient Sparse Mapping and Localization using Point, Line and Plane for Monocular, RGB-D and Stereo Cameras [13.693353009049773]
  This paper demonstrates a visual SLAM system that utilizes point and line cloud for robust camera localization, simultaneously, with an embedded piece-wise planar reconstruction (PPR) module. We address the challenge of reconstructing geometric primitives with scale ambiguity by proposing several run-time optimizations on the reconstructed lines and planes. The results show that our proposed SLAM tightly incorporates the semantic features to boost both tracking as well as backend optimization.
  arXiv  Detail & Related papers  (2022-07-13T09:05:35Z)
• Improving Monocular Visual Odometry Using Learned Depth [84.05081552443693]
  We propose a framework to exploit monocular depth estimation for improving visual odometry (VO) The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes. Compared with current learning-based VO methods, our method demonstrates a stronger generalization ability to diverse scenes.
  arXiv  Detail & Related papers  (2022-04-04T06:26:46Z)
• SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning [53.78813049373321]
  We propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation. Our approach is useful for various applications such as mobile robot navigation and is applicable to diverse environments.
  arXiv  Detail & Related papers  (2022-03-10T12:28:42Z)
• NICE-SLAM: Neural Implicit Scalable Encoding for SLAM [112.6093688226293]
  NICE-SLAM is a dense SLAM system that incorporates multi-level local information by introducing a hierarchical scene representation. Compared to recent neural implicit SLAM systems, our approach is more scalable, efficient, and robust.
  arXiv  Detail & Related papers  (2021-12-22T18:45:44Z)
• A Hybrid Sparse-Dense Monocular SLAM System for Autonomous Driving [0.5735035463793008]
  We reconstruct a dense 3D model of the geometry of an outdoor environment using a single monocular camera attached to a moving vehicle. Our system employs dense depth prediction with a hybrid mapping architecture combining state-of-the-art sparse features and dense fusion-based visual SLAM algorithms.
  arXiv  Detail & Related papers  (2021-08-17T16:13:01Z)
• CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations [20.79223452551813]
  State-of-the-art sparse visual SLAM systems provide accurate estimates of the camera trajectory and locations of landmarks. While these sparse maps are useful for localization, they cannot be used for other tasks such as obstacle avoidance or scene understanding. We propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses and sparse points produced by the SLAM system and predicts a dense depth image for every.
  arXiv  Detail & Related papers  (2021-07-19T16:13:18Z)
• Pseudo RGB-D for Self-Improving Monocular SLAM and Depth Prediction [72.30870535815258]
  CNNs for monocular depth prediction represent two largely disjoint approaches towards building a 3D map of the surrounding environment. We propose a joint narrow and wide baseline based self-improving framework, where on the one hand the CNN-predicted depth is leveraged to perform pseudo RGB-D feature-based SLAM. On the other hand, the bundle-adjusted 3D scene structures and camera poses from the more principled geometric SLAM are injected back into the depth network through novel wide baseline losses.
  arXiv  Detail & Related papers  (2020-04-22T16:31:59Z)
• OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems [88.41004332322788]
  We present an omnidirectional localization and dense mapping system for a wide-baseline multiview stereo setup with ultra-wide field-of-view (FOV) fisheye cameras. For more practical and accurate reconstruction, we first introduce improved and light-weighted deep neural networks for the omnidirectional depth estimation. We integrate our omnidirectional depth estimates into the visual odometry (VO) and add a loop closing module for global consistency.
  arXiv  Detail & Related papers  (2020-03-18T05:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.