OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration

• URL: http://arxiv.org/abs/2411.19278v1
• Date: Thu, 28 Nov 2024 17:20:04 GMT
• Title: OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration
• Authors: Yiming Zuo, Willow Yang, Zeyu Ma, Jia Deng,
• Abstract summary: Depth completion (DC) aims to predict a dense depth map from an RGB image and sparse depth observations. Existing methods for DC generalize poorly on new datasets or unseen sparse depth patterns. We propose OMNI-DC, a highly robust DC model that generalizes well across various scenarios.
• Score: 26.6801726990372
• License:
• Abstract: Depth completion (DC) aims to predict a dense depth map from an RGB image and sparse depth observations. Existing methods for DC generalize poorly on new datasets or unseen sparse depth patterns, limiting their practical applications. We propose OMNI-DC, a highly robust DC model that generalizes well across various scenarios. Our method incorporates a novel multi-resolution depth integration layer and a probability-based loss, enabling it to deal with sparse depth maps of varying densities. Moreover, we train OMNI-DC on a mixture of synthetic datasets with a scale normalization technique. To evaluate our model, we establish a new evaluation protocol named Robust-DC for zero-shot testing under various sparse depth patterns. Experimental results on Robust-DC and conventional benchmarks show that OMNI-DC significantly outperforms the previous state of the art. The checkpoints, training code, and evaluations are available at https://github.com/princeton-vl/OMNI-DC.

Related papers

• OGNI-DC: Robust Depth Completion with Optimization-Guided Neural Iterations [23.0962036039182]
  "Optimization-Guided Neural Iterations" (OGNI) is a novel framework for depth completion. OGNI-DC exhibits strong generalization, outperforming baselines on unseen datasets and across various sparsity levels. It has high accuracy, achieving state-of-the-art performance on the NYUv2 and the KITTI benchmarks.
  arXiv  Detail & Related papers  (2024-06-17T16:30:29Z)
• Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark [101.23684938489413]
  Anomaly detection (AD) is often focused on detecting anomalies for industrial quality inspection and medical lesion examination. This work first constructs a large-scale and general-purpose COCO-AD dataset by extending COCO to the AD field. Inspired by the metrics in the segmentation field, we propose several more practical threshold-dependent AD-specific metrics.
  arXiv  Detail & Related papers  (2024-04-16T17:38:26Z)
• SparseDC: Depth Completion from sparse and non-uniform inputs [18.20396821395775]
  We propose SparseDC, a model for Depth Completion of Sparse and non-uniform depth inputs. The key contributions of SparseDC are two-fold. First, we design a simple strategy, called SFFM, to improve the robustness under sparse input. Second, we propose a two-branch feature embedder to predict both the precise local geometry of regions with available depth values and accurate structures in regions with no depth.
  arXiv  Detail & Related papers  (2023-11-30T13:36:27Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• Monocular Visual-Inertial Depth Estimation [66.71452943981558]
  We present a visual-inertial depth estimation pipeline that integrates monocular depth estimation and visual-inertial odometry. Our approach performs global scale and shift alignment against sparse metric depth, followed by learning-based dense alignment. We evaluate on the TartanAir and VOID datasets, observing up to 30% reduction in RMSE with dense scale alignment.
  arXiv  Detail & Related papers  (2023-03-21T18:47:34Z)
• Deep Combinatorial Aggregation [58.78692706974121]
  Deep ensemble is a simple and effective method that achieves state-of-the-art results for uncertainty-aware learning tasks. In this work, we explore a generalization of deep ensemble called deep aggregation (DCA) DCA creates multiple instances of network components and aggregates their combinations to produce diversified model proposals and predictions.
  arXiv  Detail & Related papers  (2022-10-12T17:35:03Z)
• Towards Domain-agnostic Depth Completion [28.25756709062647]
  Existing depth completion methods are often targeted at a specific sparse depth type and generalize poorly across task domains. We present a method to complete sparse/semi-dense, noisy, and potentially low-resolution depth maps obtained by various range sensors. Our method shows superior cross-domain generalization ability against state-of-the-art depth completion methods.
  arXiv  Detail & Related papers  (2022-07-29T04:10:22Z)
• AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks [78.62086125399831]
  We present a general approach called Alternating Compressed/DeCompressed (AC/DC) training of deep neural networks (DNNs) AC/DC outperforms existing sparse training methods in accuracy at similar computational budgets. An important property of AC/DC is that it allows co-training of dense and sparse models, yielding accurate sparse-dense model pairs at the end of the training process.
  arXiv  Detail & Related papers  (2021-06-23T13:23:00Z)
• Densely Nested Top-Down Flows for Salient Object Detection [137.74130900326833]
  This paper revisits the role of top-down modeling in salient object detection. It designs a novel densely nested top-down flows (DNTDF)-based framework. In every stage of DNTDF, features from higher levels are read in via the progressive compression shortcut paths (PCSP)
  arXiv  Detail & Related papers  (2021-02-18T03:14:02Z)
• AdaBins: Depth Estimation using Adaptive Bins [43.07310038858445]
  We propose a transformer-based architecture block that divides the depth range into bins whose center value is estimated adaptively per image. Our results show a decisive improvement over the state-of-the-art on several popular depth datasets.
  arXiv  Detail & Related papers  (2020-11-28T14:40:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.