SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation

• URL: http://arxiv.org/abs/2411.18229v1
• Date: Wed, 27 Nov 2024 11:07:27 GMT
• Title: SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation
• Authors: Duc-Hai Pham, Tung Do, Phong Nguyen, Binh-Son Hua, Khoi Nguyen, Rang Nguyen,
• Abstract summary: SharpDepth combines the metric accuracy of discriminative depth estimation methods with the fine-grained boundary sharpness typically achieved by generative methods. Our approach bridges these limitations by integrating metric accuracy with detailed boundary preservation, resulting in depth predictions that are both metrically precise and visually sharp.
• Score: 17.88798247412388
• License:
• Abstract: We propose SharpDepth, a novel approach to monocular metric depth estimation that combines the metric accuracy of discriminative depth estimation methods (e.g., Metric3D, UniDepth) with the fine-grained boundary sharpness typically achieved by generative methods (e.g., Marigold, Lotus). Traditional discriminative models trained on real-world data with sparse ground-truth depth can accurately predict metric depth but often produce over-smoothed or low-detail depth maps. Generative models, in contrast, are trained on synthetic data with dense ground truth, generating depth maps with sharp boundaries yet only providing relative depth with low accuracy. Our approach bridges these limitations by integrating metric accuracy with detailed boundary preservation, resulting in depth predictions that are both metrically precise and visually sharp. Our extensive zero-shot evaluations on standard depth estimation benchmarks confirm SharpDepth effectiveness, showing its ability to achieve both high depth accuracy and detailed representation, making it well-suited for applications requiring high-quality depth perception across diverse, real-world environments.

Related papers

• Revisiting Gradient-based Uncertainty for Monocular Depth Estimation [10.502852645001882]
  We introduce gradient-based uncertainty estimation for monocular depth estimation models. We demonstrate that our approach is effective in determining the uncertainty without re-training. In particular, for models trained with monocular sequences and therefore most prone to uncertainty, our method outperforms related approaches.
  arXiv  Detail & Related papers  (2025-02-09T17:21:41Z)
• Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion [51.69876947593144]
  Existing methods for depth completion operate in tightly constrained settings. Inspired by advances in monocular depth estimation, we reframe depth completion as an image-conditional depth map generation. Marigold-DC builds on a pretrained latent diffusion model for monocular depth estimation and injects the depth observations as test-time guidance.
  arXiv  Detail & Related papers  (2024-12-18T00:06:41Z)
• MetricGold: Leveraging Text-To-Image Latent Diffusion Models for Metric Depth Estimation [9.639797094021988]
  MetricGold is a novel approach that harnesses generative diffusion model's rich priors to improve metric depth estimation. Our experiments demonstrate robust generalization across diverse datasets, producing sharper and higher quality metric depth estimates.
  arXiv  Detail & Related papers  (2024-11-16T20:59:01Z)
• Depth Pro: Sharp Monocular Metric Depth in Less Than a Second [45.6690958201871]
  We present a foundation model for zero-shot metric monocular depth estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with unparalleled sharpness and high-frequency details. It produces a 2.25-megapixel depth map in 0.3 seconds on a standard GPU.
  arXiv  Detail & Related papers  (2024-10-02T22:42:20Z)
• ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation [62.600382533322325]
  We propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction module. Our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework.
  arXiv  Detail & Related papers  (2024-07-11T05:11:56Z)
• Monocular Visual-Inertial Depth Estimation [66.71452943981558]
  We present a visual-inertial depth estimation pipeline that integrates monocular depth estimation and visual-inertial odometry. Our approach performs global scale and shift alignment against sparse metric depth, followed by learning-based dense alignment. We evaluate on the TartanAir and VOID datasets, observing up to 30% reduction in RMSE with dense scale alignment.
  arXiv  Detail & Related papers  (2023-03-21T18:47:34Z)
• Probabilistic Volumetric Fusion for Dense Monocular SLAM [33.156523309257786]
  We present a novel method to reconstruct 3D scenes by leveraging deep dense monocular SLAM and fast uncertainty propagation. The proposed approach is able to 3D reconstruct scenes densely, accurately, and in real-time while being robust to extremely noisy depth estimates. We show that our approach achieves 92% better accuracy than directly fusing depths from monocular SLAM, and up to 90% improvements compared to the best competing approach.
  arXiv  Detail & Related papers  (2022-10-03T23:53:35Z)
• Variational Monocular Depth Estimation for Reliability Prediction [12.951621755732544]
  Self-supervised learning for monocular depth estimation is widely investigated as an alternative to supervised learning approach. Previous works have successfully improved the accuracy of depth estimation by modifying the model structure. In this paper, we theoretically formulate a variational model for the monocular depth estimation to predict the reliability of the estimated depth image.
  arXiv  Detail & Related papers  (2020-11-24T06:23:51Z)
• Adaptive confidence thresholding for monocular depth estimation [83.06265443599521]
  We propose a new approach to leverage pseudo ground truth depth maps of stereo images generated from self-supervised stereo matching methods. The confidence map of the pseudo ground truth depth map is estimated to mitigate performance degeneration by inaccurate pseudo depth maps. Experimental results demonstrate superior performance to state-of-the-art monocular depth estimation methods.
  arXiv  Detail & Related papers  (2020-09-27T13:26:16Z)
• Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
  We present a new learning-based method for multi-frame depth estimation from a color video. Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-02T07:10:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.