Online Adaptation of Monocular Depth Prediction with Visual SLAM

• URL: http://arxiv.org/abs/2111.04096v1
• Date: Sun, 7 Nov 2021 14:20:35 GMT
• Title: Online Adaptation of Monocular Depth Prediction with Visual SLAM
• Authors: Shing Yan Loo, Moein Shakeri, Sai Hong Tang, Syamsiah Mashohor, Hong Zhang
• Abstract summary: The ability of accurate depth prediction by a CNN is a major challenge for its wide use in practical visual SLAM applications. We propose a novel online adaptation framework consisting of two complementary processes to fine-tune the depth prediction. Experimental results on both benchmark datasets and a real robot in our own experimental environments show that our proposed method improves the SLAM reconstruction accuracy.
• Score: 8.478040209440868
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ability of accurate depth prediction by a CNN is a major challenge for its wide use in practical visual SLAM applications, such as enhanced camera tracking and dense mapping. This paper is set out to answer the following question: Can we tune a depth prediction CNN with the help of a visual SLAM algorithm even if the CNN is not trained for the current operating environment in order to benefit the SLAM performance? To this end, we propose a novel online adaptation framework consisting of two complementary processes: a SLAM algorithm that is used to generate keyframes to fine-tune the depth prediction and another algorithm that uses the online adapted depth to improve map quality. Once the potential noisy map points are removed, we perform global photometric bundle adjustment (BA) to improve the overall SLAM performance. Experimental results on both benchmark datasets and a real robot in our own experimental environments show that our proposed method improves the SLAM reconstruction accuracy. We demonstrate the use of regularization in the training loss as an effective means to prevent catastrophic forgetting. In addition, we compare our online adaptation framework against the state-of-the-art pre-trained depth prediction CNNs to show that our online adapted depth prediction CNN outperforms the depth prediction CNNs that have been trained on a large collection of datasets.

Related papers

• UncLe-SLAM: Uncertainty Learning for Dense Neural SLAM [60.575435353047304]
  We present an uncertainty learning framework for dense neural simultaneous localization and mapping (SLAM) We propose an online framework for sensor uncertainty estimation that can be trained in a self-supervised manner from only 2D input data.
  arXiv  Detail & Related papers  (2023-06-19T16:26:25Z)
• CNN-Augmented Visual-Inertial SLAM with Planar Constraints [26.024485121674328]
  We present a robust visual-inertial SLAM system that combines the benefits of Convolutional Neural Networks (CNNs) and planar constraints. We use a CNN to predict the depth map and the corresponding uncertainty map for each image. We also present a fast plane detection method that detects horizontal planes via one-point RANSAC and vertical planes via two-point RANSAC.
  arXiv  Detail & Related papers  (2022-05-05T21:49:57Z)
• 3DVNet: Multi-View Depth Prediction and Volumetric Refinement [68.68537312256144]
  3DVNet is a novel multi-view stereo (MVS) depth-prediction method. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions. We show that our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics.
  arXiv  Detail & Related papers  (2021-12-01T00:52:42Z)
• Application of 2-D Convolutional Neural Networks for Damage Detection in Steel Frame Structures [0.0]
  We present an application of 2-D convolutional neural networks (2-D CNNs) designed to perform both feature extraction and classification stages. The method uses a network of lighted CNNs instead of deep and takes raw acceleration signals as input.
  arXiv  Detail & Related papers  (2021-10-29T16:29:31Z)
• A Front-End for Dense Monocular SLAM using a Learned Outlier Mask Prior [11.468537169201083]
  Recent achievements in depth prediction from a single RGB image have powered the new research area of combining convolutional neural networks (CNNs) with classical simultaneous localization and mapping (SLAM) algorithms. Most of the current CNN-SLAM approaches have only taken advantage of the depth prediction but not yet other products from a CNN. We devise a dense CNN-assisted SLAM front-end that is implementable with sparse and evaluate it on both indoor and outdoor datasets.
  arXiv  Detail & Related papers  (2021-04-01T15:43:28Z)
• BreakingBED -- Breaking Binary and Efficient Deep Neural Networks by Adversarial Attacks [65.2021953284622]
  We study robustness of CNNs against white-box and black-box adversarial attacks. Results are shown for distilled CNNs, agent-based state-of-the-art pruned models, and binarized neural networks.
  arXiv  Detail & Related papers  (2021-03-14T20:43:19Z)
• RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr [60.07531696857743]
  Fine-tuning the deep convolution neural network(CNN) using a pre-trained model helps transfer knowledge learned from larger datasets to the target task. We propose RIFLE - a strategy that deepens backpropagation in transfer learning settings. RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning.
  arXiv  Detail & Related papers  (2020-07-07T11:27:43Z)
• Cascaded Deep Video Deblurring Using Temporal Sharpness Prior [88.98348546566675]
  The proposed algorithm mainly consists of optical flow estimation from intermediate latent frames and latent frame restoration steps. It first develops a deep CNN model to estimate optical flow from intermediate latent frames and then restores the latent frames based on the estimated optical flow. We show that exploring the domain knowledge of video deblurring is able to make the deep CNN model more compact and efficient.
  arXiv  Detail & Related papers  (2020-04-06T09:13:49Z)
• What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective [102.37204254403038]
  We make an attempt to understand what deep CNNs benefit from GCP in a viewpoint of optimization. We show that GCP can make the optimization landscape more smooth and the gradients more predictive. We conduct extensive experiments using various deep CNN models on diversified tasks, and the results provide strong support to our findings.
  arXiv  Detail & Related papers  (2020-03-25T07:00:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.