ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation

• URL: http://arxiv.org/abs/2407.08187v1
• Date: Thu, 11 Jul 2024 05:11:56 GMT
• Title: ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation
• Authors: Ruijie Zhu, Chuxin Wang, Ziyang Song, Li Liu, Tianzhu Zhang, Yongdong Zhang,
• Abstract summary: We propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction module. Our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework.
• Score: 62.600382533322325
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Estimating depth from a single image is a challenging visual task. Compared to relative depth estimation, metric depth estimation attracts more attention due to its practical physical significance and critical applications in real-life scenarios. However, existing metric depth estimation methods are typically trained on specific datasets with similar scenes, facing challenges in generalizing across scenes with significant scale variations. To address this challenge, we propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction (SASP) module and an adaptive relative depth estimation (ARDE) module, respectively. The proposed ScaleDepth enjoys several merits. First, the SASP module can implicitly combine structural and semantic features of the images to predict precise scene scales. Second, the ARDE module can adaptively estimate the relative depth distribution of each image within a normalized depth space. Third, our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework, without the need for setting the depth range or fine-tuning model. Extensive experiments demonstrate that our method attains state-of-the-art performance across indoor, outdoor, unconstrained, and unseen scenes. Project page: https://ruijiezhu94.github.io/ScaleDepth

Related papers

• Learning to Adapt CLIP for Few-Shot Monocular Depth Estimation [31.34615135846137]
  We propose a few-shot-based method which learns to adapt the Vision-Language Models for monocular depth estimation. Specifically, it assigns different depth bins for different scenes, which can be selected by the model during inference. With only one image per scene for training, our extensive experiment results on the NYU V2 and KITTI dataset demonstrate that our method outperforms the previous state-of-the-art method by up to 10.6% in terms of MARE.
  arXiv  Detail & Related papers  (2023-11-02T06:56:50Z)
• Blur aware metric depth estimation with multi-focus plenoptic cameras [8.508198765617196]
  We present a new metric depth estimation algorithm using only raw images from a multi-focus plenoptic camera. The proposed approach is especially suited for the multi-focus configuration where several micro-lenses with different focal lengths are used.
  arXiv  Detail & Related papers  (2023-08-08T13:38:50Z)
• FS-Depth: Focal-and-Scale Depth Estimation from a Single Image in Unseen Indoor Scene [57.26600120397529]
  It has long been an ill-posed problem to predict absolute depth maps from single images in real (unseen) indoor scenes. We develop a focal-and-scale depth estimation model to well learn absolute depth maps from single images in unseen indoor scenes.
  arXiv  Detail & Related papers  (2023-07-27T04:49:36Z)
• Self-Supervised Learning based Depth Estimation from Monocular Images [0.0]
  The goal of Monocular Depth Estimation is to predict the depth map, given a 2D monocular RGB image as input. We plan to do intrinsic camera parameters during training and apply weather augmentations to further generalize our model.
  arXiv  Detail & Related papers  (2023-04-14T07:14:08Z)
• Monocular Visual-Inertial Depth Estimation [66.71452943981558]
  We present a visual-inertial depth estimation pipeline that integrates monocular depth estimation and visual-inertial odometry. Our approach performs global scale and shift alignment against sparse metric depth, followed by learning-based dense alignment. We evaluate on the TartanAir and VOID datasets, observing up to 30% reduction in RMSE with dense scale alignment.
  arXiv  Detail & Related papers  (2023-03-21T18:47:34Z)
• SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes [58.89295356901823]
  Self-supervised monocular depth estimation has shown impressive results in static scenes. It relies on the multi-view consistency assumption for training networks, however, that is violated in dynamic object regions. We introduce an external pretrained monocular depth estimation model for generating single-image depth prior. Our model can predict sharp and accurate depth maps, even when training from monocular videos of highly-dynamic scenes.
  arXiv  Detail & Related papers  (2022-11-07T16:17:47Z)
• Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image [91.71077190961688]
  We propose a two-stage framework that first predicts depth up to an unknown scale and shift from a single monocular image. We then exploits 3D point cloud data to predict the depth shift and the camera's focal length that allow us to recover 3D scene shapes. We test our depth model on nine unseen datasets and achieve state-of-the-art performance on zero-shot evaluation.
  arXiv  Detail & Related papers  (2022-08-28T16:20:14Z)
• MonoIndoor++:Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments [45.89629401768049]
  Self-supervised monocular depth estimation has seen significant progress in recent years, especially in outdoor environments. However, depth prediction results are not satisfying in indoor scenes where most of the existing data are captured with hand-held devices. We propose a novel framework-IndoorMono++ to improve the performance of self-supervised monocular depth estimation for indoor environments.
  arXiv  Detail & Related papers  (2022-07-18T21:34:43Z)
• Improving Monocular Visual Odometry Using Learned Depth [84.05081552443693]
  We propose a framework to exploit monocular depth estimation for improving visual odometry (VO) The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes. Compared with current learning-based VO methods, our method demonstrates a stronger generalization ability to diverse scenes.
  arXiv  Detail & Related papers  (2022-04-04T06:26:46Z)
• Improving Depth Estimation using Location Information [0.0]
  This paper improves the self-supervised deep learning techniques to perform accurate generalized monocular depth estimation. The main idea is to train the deep model to take into account a sequence of the different frames, each frame is geotagged with its location information.
  arXiv  Detail & Related papers  (2021-12-27T22:30:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.