DEUX: Active Exploration for Learning Unsupervised Depth Perception

• URL: http://arxiv.org/abs/2310.06164v1
• Date: Sat, 16 Sep 2023 23:33:15 GMT
• Title: DEUX: Active Exploration for Learning Unsupervised Depth Perception
• Authors: Marvin Chanc\'an, Alex Wong, Ian Abraham
• Abstract summary: We develop an active, task-informed, depth uncertainty-based motion planning approach for learning depth completion. We show that our approach further improves zero-shot generalization, while offering new insights into integrating robot learning-based depth estimation.
• Score: 8.044217507775999
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Depth perception models are typically trained on non-interactive datasets with predefined camera trajectories. However, this often introduces systematic biases into the learning process correlated to specific camera paths chosen during data acquisition. In this paper, we investigate the role of how data is collected for learning depth completion, from a robot navigation perspective, by leveraging 3D interactive environments. First, we evaluate four depth completion models trained on data collected using conventional navigation techniques. Our key insight is that existing exploration paradigms do not necessarily provide task-specific data points to achieve competent unsupervised depth completion learning. We then find that data collected with respect to photometric reconstruction has a direct positive influence on model performance. As a result, we develop an active, task-informed, depth uncertainty-based motion planning approach for learning depth completion, which we call DEpth Uncertainty-guided eXploration (DEUX). Training with data collected by our approach improves depth completion by an average greater than 18% across four depth completion models compared to existing exploration methods on the MP3D test set. We show that our approach further improves zero-shot generalization, while offering new insights into integrating robot learning-based depth estimation.

Related papers

• UnCLe: Unsupervised Continual Learning of Depth Completion [5.677777151863184]
  UnCLe is a standardized benchmark for Unsupervised Continual Learning of a multimodal depth estimation task. We benchmark depth completion models under the practical scenario of unsupervised learning over continuous streams of data.
  arXiv  Detail & Related papers  (2024-10-23T17:56:33Z)
• Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understanding [50.448520056844885]
  We propose a generative Bayesian network to produce diverse synthetic scenes with real-world patterns. A series of experiments robustly display our method's consistent superiority over existing state-of-the-art pre-training approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:43:53Z)
• Depth-discriminative Metric Learning for Monocular 3D Object Detection [14.554132525651868]
  We introduce a novel metric learning scheme that encourages the model to extract depth-discriminative features regardless of the visual attributes. Our method consistently improves the performance of various baselines by 23.51% and 5.78% on average.
  arXiv  Detail & Related papers  (2024-01-02T07:34:09Z)
• Self-Supervised Depth Completion Guided by 3D Perception and Geometry Consistency [17.68427514090938]
  This paper explores the utilization of 3D perceptual features and multi-view geometry consistency to devise a high-precision self-supervised depth completion method. Experiments on benchmark datasets of NYU-Depthv2 and VOID demonstrate that the proposed model achieves the state-of-the-art depth completion performance.
  arXiv  Detail & Related papers  (2023-12-23T14:19:56Z)
• Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering [93.94371335579321]
  We propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations. Comprehensive experiments underscore our framework's superior generalization capabilities. Our innovative loss functions empower the model to autonomously recover domain-specific scale-and-shift coefficients.
  arXiv  Detail & Related papers  (2023-09-18T12:36:39Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes [58.89295356901823]
  Self-supervised monocular depth estimation has shown impressive results in static scenes. It relies on the multi-view consistency assumption for training networks, however, that is violated in dynamic object regions. We introduce an external pretrained monocular depth estimation model for generating single-image depth prior. Our model can predict sharp and accurate depth maps, even when training from monocular videos of highly-dynamic scenes.
  arXiv  Detail & Related papers  (2022-11-07T16:17:47Z)
• Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking [47.59619420444781]
  Approaches to monocular 3D perception including detection and tracking often yield inferior performance when compared to LiDAR-based techniques. We propose a multi-level fusion method that combines different representations (RGB and pseudo-LiDAR) and temporal information across multiple frames for objects (tracklets) to enhance per-object depth estimation.
  arXiv  Detail & Related papers  (2022-06-08T03:37:59Z)
• Unsupervised Single-shot Depth Estimation using Perceptual Reconstruction [0.0]
  This study presents the most recent advances in the field of generative neural networks, leveraging them to perform fully unsupervised single-shot depth synthesis. Two generators for RGB-to-depth and depth-to-RGB transfer are implemented and simultaneously optimized using the Wasserstein-1 distance and a novel perceptual reconstruction term. The success observed in this study suggests the great potential for unsupervised single-shot depth estimation in real-world applications.
  arXiv  Detail & Related papers  (2022-01-28T15:11:34Z)
• PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding [107.02479689909164]
  In this work, we aim at facilitating research on 3D representation learning. We measure the effect of unsupervised pre-training on a large source set of 3D scenes.
  arXiv  Detail & Related papers  (2020-07-21T17:59:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.