UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler

• URL: http://arxiv.org/abs/2502.20110v1
• Date: Thu, 27 Feb 2025 14:03:15 GMT
• Title: UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler
• Authors: Luigi Piccinelli, Christos Sakaridis, Yung-Hsu Yang, Mattia Segu, Siyuan Li, Wim Abbeloos, Luc Van Gool,
• Abstract summary: We propose a new model, UniDepthV2, capable of reconstructing metric 3D scenes from solely single images across domains.<n>UniDepthV2 directly predicts metric 3D points from the input image at inference time without any additional information.<n>Our model exploits a pseudo-spherical output representation, which disentangles the camera and depth representations.
• Score: 62.06785782635153
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurate monocular metric depth estimation (MMDE) is crucial to solving downstream tasks in 3D perception and modeling. However, the remarkable accuracy of recent MMDE methods is confined to their training domains. These methods fail to generalize to unseen domains even in the presence of moderate domain gaps, which hinders their practical applicability. We propose a new model, UniDepthV2, capable of reconstructing metric 3D scenes from solely single images across domains. Departing from the existing MMDE paradigm, UniDepthV2 directly predicts metric 3D points from the input image at inference time without any additional information, striving for a universal and flexible MMDE solution. In particular, UniDepthV2 implements a self-promptable camera module predicting a dense camera representation to condition depth features. Our model exploits a pseudo-spherical output representation, which disentangles the camera and depth representations. In addition, we propose a geometric invariance loss that promotes the invariance of camera-prompted depth features. UniDepthV2 improves its predecessor UniDepth model via a new edge-guided loss which enhances the localization and sharpness of edges in the metric depth outputs, a revisited, simplified and more efficient architectural design, and an additional uncertainty-level output which enables downstream tasks requiring confidence. Thorough evaluations on ten depth datasets in a zero-shot regime consistently demonstrate the superior performance and generalization of UniDepthV2. Code and models are available at https://github.com/lpiccinelli-eth/UniDepth

Related papers

• Learning A Zero-shot Occupancy Network from Vision Foundation Models via Self-supervised Adaptation [41.98740330990215]
  This work proposes a novel approach that bridges 2D vision foundation models with 3D tasks. We leverage the zero-shot capabilities of vision-language models for image semantics. We project the semantics into 3D space using the reconstructed metric depth, thereby providing 3D supervision.
  arXiv  Detail & Related papers  (2025-03-10T09:54:40Z)
• Enhanced Encoder-Decoder Architecture for Accurate Monocular Depth Estimation [0.0]
  This paper introduces a novel deep learning-based approach using an enhanced encoder-decoder architecture.<n>It incorporates multi-scale feature extraction to enhance depth prediction accuracy across various object sizes and distances.<n> Experimental results on the KITTI dataset show that our model achieves a significantly faster inference time of 0.019 seconds.
  arXiv  Detail & Related papers  (2024-10-15T13:46:19Z)
• SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learning [17.99904937160487]
  We introduce SCIPaD, a novel approach that incorporates spatial clues for unsupervised depth-pose joint learning. SCIPaD achieves a reduction of 22.2% in average translation error and 34.8% in average angular error for camera pose estimation task on the KITTI Odometry dataset.
  arXiv  Detail & Related papers  (2024-07-07T06:52:51Z)
• UniDepth: Universal Monocular Metric Depth Estimation [81.80512457953903]
  We propose a new model, UniDepth, capable of reconstructing metric 3D scenes from solely single images across domains. Our model exploits a pseudo-spherical output representation, which disentangles camera and depth representations. Thorough evaluations on ten datasets in a zero-shot regime consistently demonstrate the superior performance of UniDepth.
  arXiv  Detail & Related papers  (2024-03-27T18:06:31Z)
• Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation [74.28509379811084]
  Metric3D v2 is a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image.<n>We propose solutions for both metric depth estimation and surface normal estimation.<n>Our method enables the accurate recovery of metric 3D structures on randomly collected internet images.
  arXiv  Detail & Related papers  (2024-03-22T02:30:46Z)
• Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image [85.91935485902708]
  We show that the key to a zero-shot single-view metric depth model lies in the combination of large-scale data training and resolving the metric ambiguity from various camera models. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problems and can be effortlessly plugged into existing monocular models. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images.
  arXiv  Detail & Related papers  (2023-07-20T16:14:23Z)
• Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
  We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
  arXiv  Detail & Related papers  (2023-03-31T16:01:03Z)
• Towards Domain Generalization for Multi-view 3D Object Detection in Bird-Eye-View [11.958753088613637]
  We first analyze the causes of the domain gap for the MV3D-Det task. To acquire a robust depth prediction, we propose to decouple the depth estimation from intrinsic parameters of the camera. We modify the focal length values to create multiple pseudo-domains and construct an adversarial training loss to encourage the feature representation to be more domain-agnostic.
  arXiv  Detail & Related papers  (2023-03-03T02:59:13Z)
• DeepMLE: A Robust Deep Maximum Likelihood Estimator for Two-view Structure from Motion [9.294501649791016]
  Two-view structure from motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM (vSLAM) We formulate the two-view SfM problem as a maximum likelihood estimation (MLE) and solve it with the proposed framework, denoted as DeepMLE. Our method significantly outperforms the state-of-the-art end-to-end two-view SfM approaches in accuracy and generalization capability.
  arXiv  Detail & Related papers  (2022-10-11T15:07:25Z)
• Towards Model Generalization for Monocular 3D Object Detection [57.25828870799331]
  We present an effective unified camera-generalized paradigm (CGP) for Mono3D object detection. We also propose the 2D-3D geometry-consistent object scaling strategy (GCOS) to bridge the gap via an instance-level augment. Our method called DGMono3D achieves remarkable performance on all evaluated datasets and surpasses the SoTA unsupervised domain adaptation scheme.
  arXiv  Detail & Related papers  (2022-05-23T23:05:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.