UnCLe: Unsupervised Continual Learning of Depth Completion

• URL: http://arxiv.org/abs/2410.18074v2
• Date: Fri, 25 Oct 2024 17:37:29 GMT
• Title: UnCLe: Unsupervised Continual Learning of Depth Completion
• Authors: Suchisrit Gangopadhyay, Xien Chen, Michael Chu, Patrick Rim, Hyoungseob Park, Alex Wong,
• Abstract summary: UnCLe is a standardized benchmark for Unsupervised Continual Learning of a multimodal depth estimation task. We benchmark depth completion models under the practical scenario of unsupervised learning over continuous streams of data.
• Score: 5.677777151863184
• License:
• Abstract: We propose UnCLe, a standardized benchmark for Unsupervised Continual Learning of a multimodal depth estimation task: Depth completion aims to infer a dense depth map from a pair of synchronized RGB image and sparse depth map. We benchmark depth completion models under the practical scenario of unsupervised learning over continuous streams of data. Existing methods are typically trained on a static, or stationary, dataset. However, when adapting to novel non-stationary distributions, they "catastrophically forget" previously learned information. UnCLe simulates these non-stationary distributions by adapting depth completion models to sequences of datasets containing diverse scenes captured from distinct domains using different visual and range sensors. We adopt representative methods from continual learning paradigms and translate them to enable unsupervised continual learning of depth completion. We benchmark these models for indoor and outdoor and investigate the degree of catastrophic forgetting through standard quantitative metrics. Furthermore, we introduce model inversion quality as an additional measure of forgetting. We find that unsupervised continual learning of depth completion is an open problem, and we invite researchers to leverage UnCLe as a development platform.

Related papers

• Continual Learning of Unsupervised Monocular Depth from Videos [19.43053045216986]
  We introduce a framework that captures challenges of continual unsupervised depth estimation (CUDE) We propose a rehearsal-based dual-memory method, MonoDepthCL, which utilizes collected ontemporal consistency for continual learning in depth estimation.
  arXiv  Detail & Related papers  (2023-11-04T12:36:07Z)
• Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering [93.94371335579321]
  We propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations. Comprehensive experiments underscore our framework's superior generalization capabilities. Our innovative loss functions empower the model to autonomously recover domain-specific scale-and-shift coefficients.
  arXiv  Detail & Related papers  (2023-09-18T12:36:39Z)
• DEUX: Active Exploration for Learning Unsupervised Depth Perception [8.044217507775999]
  We develop an active, task-informed, depth uncertainty-based motion planning approach for learning depth completion. We show that our approach further improves zero-shot generalization, while offering new insights into integrating robot learning-based depth estimation.
  arXiv  Detail & Related papers  (2023-09-16T23:33:15Z)
• SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes [58.89295356901823]
  Self-supervised monocular depth estimation has shown impressive results in static scenes. It relies on the multi-view consistency assumption for training networks, however, that is violated in dynamic object regions. We introduce an external pretrained monocular depth estimation model for generating single-image depth prior. Our model can predict sharp and accurate depth maps, even when training from monocular videos of highly-dynamic scenes.
  arXiv  Detail & Related papers  (2022-11-07T16:17:47Z)
• SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning [53.78813049373321]
  We propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation. Our approach is useful for various applications such as mobile robot navigation and is applicable to diverse environments.
  arXiv  Detail & Related papers  (2022-03-10T12:28:42Z)
• Toward Practical Self-Supervised Monocular Indoor Depth Estimation [17.222362224649544]
  We show that self-supervised monocular depth estimation methods generalize poorly to unseen indoor scenes. We propose a structure distillation approach to learn knacks from a pretrained depth estimator that produces structured but metric-agnostic depth. By combining distillation with the self-supervised branch that learns metrics from left-right consistency, we attain structured and metric depth for generic indoor scenes.
  arXiv  Detail & Related papers  (2021-12-04T11:02:56Z)
• An Adaptive Framework for Learning Unsupervised Depth Completion [59.17364202590475]
  We present a method to infer a dense depth map from a color image and associated sparse depth measurements. We show that regularization and co-visibility are related via the fitness of the model to data and can be unified into a single framework.
  arXiv  Detail & Related papers  (2021-06-06T02:27:55Z)
• Unsupervised Scale-consistent Depth Learning from Video [131.3074342883371]
  We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training. Thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system. The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training.
  arXiv  Detail & Related papers  (2021-05-25T02:17:56Z)
• Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction [87.08227378010874]
  We show the importance of the high-order 3D geometric constraints for depth prediction. By designing a loss term that enforces a simple geometric constraint, we significantly improve the accuracy and robustness of monocular depth estimation. We show state-of-the-art results of learning metric depth on NYU Depth-V2 and KITTI.
  arXiv  Detail & Related papers  (2021-03-07T00:08:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.