Generalizing to the Open World: Deep Visual Odometry with Online Adaptation

• URL: http://arxiv.org/abs/2103.15279v1
• Date: Mon, 29 Mar 2021 02:13:56 GMT
• Title: Generalizing to the Open World: Deep Visual Odometry with Online Adaptation
• Authors: Shunkai Li, Xin Wu, Yingdian Cao, Hongbin Zha
• Abstract summary: We propose an online adaptation framework for deep VO with the assistance of scene-agnostic geometric computations and Bayesian inference. Our method achieves state-of-the-art generalization ability among self-supervised VO methods.
• Score: 27.22639812204019
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Despite learning-based visual odometry (VO) has shown impressive results in recent years, the pretrained networks may easily collapse in unseen environments. The large domain gap between training and testing data makes them difficult to generalize to new scenes. In this paper, we propose an online adaptation framework for deep VO with the assistance of scene-agnostic geometric computations and Bayesian inference. In contrast to learning-based pose estimation, our method solves pose from optical flow and depth while the single-view depth estimation is continuously improved with new observations by online learned uncertainties. Meanwhile, an online learned photometric uncertainty is used for further depth and pose optimization by a differentiable Gauss-Newton layer. Our method enables fast adaptation of deep VO networks to unseen environments in a self-supervised manner. Extensive experiments including Cityscapes to KITTI and outdoor KITTI to indoor TUM demonstrate that our method achieves state-of-the-art generalization ability among self-supervised VO methods.

Related papers

• SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning [53.78813049373321]
  We propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation. Our approach is useful for various applications such as mobile robot navigation and is applicable to diverse environments.
  arXiv  Detail & Related papers  (2022-03-10T12:28:42Z)
• Low-light Image Enhancement by Retinex Based Algorithm Unrolling and Adjustment [50.13230641857892]
  We propose a new deep learning framework for the low-light image enhancement (LIE) problem. The proposed framework contains a decomposition network inspired by algorithm unrolling, and adjustment networks considering both global brightness and local brightness sensitivity. Experiments on a series of typical LIE datasets demonstrated the effectiveness of the proposed method, both quantitatively and visually, as compared with existing methods.
  arXiv  Detail & Related papers  (2022-02-12T03:59:38Z)
• Robust Visual Odometry Using Position-Aware Flow and Geometric Bundle Adjustment [16.04240592057438]
  A novel optical flow network (PANet) built on a position-aware mechanism is proposed first. Then, a novel system that jointly estimates depth, optical flow, and ego-motion without a typical network to learning ego-motion is proposed. Experiments show that the proposed system not only outperforms other state-of-the-art methods in terms of depth, flow, and VO estimation.
  arXiv  Detail & Related papers  (2021-11-22T12:05:27Z)
• An Adaptive Framework for Learning Unsupervised Depth Completion [59.17364202590475]
  We present a method to infer a dense depth map from a color image and associated sparse depth measurements. We show that regularization and co-visibility are related via the fitness of the model to data and can be unified into a single framework.
  arXiv  Detail & Related papers  (2021-06-06T02:27:55Z)
• Fusing the Old with the New: Learning Relative Camera Pose with Geometry-Guided Uncertainty [91.0564497403256]
  We present a novel framework that involves probabilistic fusion between the two families of predictions during network training. Our network features a self-attention graph neural network, which drives the learning by enforcing strong interactions between different correspondences. We propose motion parmeterizations suitable for learning and show that our method achieves state-of-the-art performance on the challenging DeMoN and ScanNet datasets.
  arXiv  Detail & Related papers  (2021-04-16T17:59:06Z)
• DF-VO: What Should Be Learnt for Visual Odometry? [33.379888882093965]
  We design a simple yet robust Visual Odometry system by integrating multi-view geometry and deep learning on Depth and optical Flow. Comprehensive ablation studies show the effectiveness of the proposed method, and extensive evaluation results show the state-of-the-art performance of our system.
  arXiv  Detail & Related papers  (2021-03-01T11:50:39Z)
• CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth [83.77839773394106]
  We present a lightweight, tightly-coupled deep depth network and visual-inertial odometry system. We provide the network with previously marginalized sparse features from VIO to increase the accuracy of initial depth prediction. We show that it can run in real-time with single-thread execution while utilizing GPU acceleration only for the network and code Jacobian.
  arXiv  Detail & Related papers  (2020-12-18T09:42:54Z)
• Calibrating Self-supervised Monocular Depth Estimation [77.77696851397539]
  In the recent years, many methods demonstrated the ability of neural networks to learn depth and pose changes in a sequence of images, using only self-supervision as the training signal. We show that incorporating prior information about the camera configuration and the environment, we can remove the scale ambiguity and predict depth directly, still using the self-supervised formulation and not relying on any additional sensors.
  arXiv  Detail & Related papers  (2020-09-16T14:35:45Z)
• Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas [6.1102842961275226]
  Visual odometry (VO) is a prevalent way to deal with the relative localization problem. Deep learning-based methods can automatically learn effective and robust representations. This paper aims to gain a deep insight on how deep learning can profit and optimize the VO systems.
  arXiv  Detail & Related papers  (2020-09-06T08:25:23Z)
• Self-Supervised Deep Visual Odometry with Online Adaptation [35.90781281010656]
  We propose an online meta-learning algorithm to enable VO networks to continuously adapt to new environments in a self-supervised manner. Our method consistently outperforms state-of-the-art self-supervised VO baselines considerably.
  arXiv  Detail & Related papers  (2020-05-13T03:39:29Z)
• Towards Better Generalization: Joint Depth-Pose Learning without PoseNet [36.414471128890284]
  We tackle the essential problem of scale inconsistency for self-supervised joint depth-pose learning. Most existing methods assume that a consistent scale of depth and pose can be learned across all input samples. We propose a novel system that explicitly disentangles scale from the network estimation.
  arXiv  Detail & Related papers  (2020-04-03T00:28:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.