Depth Prompting for Sensor-Agnostic Depth Estimation

• URL: http://arxiv.org/abs/2405.11867v1
• Date: Mon, 20 May 2024 08:19:08 GMT
• Title: Depth Prompting for Sensor-Agnostic Depth Estimation
• Authors: Jin-Hwi Park, Chanhwi Jeong, Junoh Lee, Hae-Gon Jeon,
• Abstract summary: We design a novel depth prompt module to allow the desirable feature representation according to new depth distributions. Our method helps the pretrained model to be free from restraint of depth scan range and to provide absolute scale depth maps.
• Score: 19.280536006736575
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Dense depth maps have been used as a key element of visual perception tasks. There have been tremendous efforts to enhance the depth quality, ranging from optimization-based to learning-based methods. Despite the remarkable progress for a long time, their applicability in the real world is limited due to systematic measurement biases such as density, sensing pattern, and scan range. It is well-known that the biases make it difficult for these methods to achieve their generalization. We observe that learning a joint representation for input modalities (e.g., images and depth), which most recent methods adopt, is sensitive to the biases. In this work, we disentangle those modalities to mitigate the biases with prompt engineering. For this, we design a novel depth prompt module to allow the desirable feature representation according to new depth distributions from either sensor types or scene configurations. Our depth prompt can be embedded into foundation models for monocular depth estimation. Through this embedding process, our method helps the pretrained model to be free from restraint of depth scan range and to provide absolute scale depth maps. We demonstrate the effectiveness of our method through extensive evaluations. Source code is publicly available at https://github.com/JinhwiPark/DepthPrompting .

Related papers

• ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation [62.600382533322325]
  We propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction module. Our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework.
  arXiv  Detail & Related papers  (2024-07-11T05:11:56Z)
• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• Towards Domain-agnostic Depth Completion [28.25756709062647]
  Existing depth completion methods are often targeted at a specific sparse depth type and generalize poorly across task domains. We present a method to complete sparse/semi-dense, noisy, and potentially low-resolution depth maps obtained by various range sensors. Our method shows superior cross-domain generalization ability against state-of-the-art depth completion methods.
  arXiv  Detail & Related papers  (2022-07-29T04:10:22Z)
• SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning [53.78813049373321]
  We propose a self-supervised learning method for the pre-trained supervised monocular depth networks to enable metrically scaled depth estimation. Our approach is useful for various applications such as mobile robot navigation and is applicable to diverse environments.
  arXiv  Detail & Related papers  (2022-03-10T12:28:42Z)
• Self-Guided Instance-Aware Network for Depth Completion and Enhancement [6.319531161477912]
  Existing methods directly interpolate the missing depth measurements based on pixel-wise image content and the corresponding neighboring depth values. We propose a novel self-guided instance-aware network (SG-IANet) that utilize self-guided mechanism to extract instance-level features that is needed for depth restoration.
  arXiv  Detail & Related papers  (2021-05-25T19:41:38Z)
• Towards Unpaired Depth Enhancement and Super-Resolution in the Wild [121.96527719530305]
  State-of-the-art data-driven methods of depth map super-resolution rely on registered pairs of low- and high-resolution depth maps of the same scenes. We consider an approach to depth map enhancement based on learning from unpaired data.
  arXiv  Detail & Related papers  (2021-05-25T16:19:16Z)
• ADAADepth: Adapting Data Augmentation and Attention for Self-Supervised Monocular Depth Estimation [8.827921242078881]
  We propose ADAA, utilising depth augmentation as depth supervision for learning accurate and robust depth. We propose a relational self-attention module that learns rich contextual features and further enhances depth results. We evaluate our predicted depth on the KITTI driving dataset and achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2021-03-01T09:06:55Z)
• Guiding Monocular Depth Estimation Using Depth-Attention Volume [38.92495189498365]
  We propose guiding depth estimation to favor planar structures that are ubiquitous especially in indoor environments. Experiments on two popular indoor datasets, NYU-Depth-v2 and ScanNet, show that our method achieves state-of-the-art depth estimation results.
  arXiv  Detail & Related papers  (2020-04-06T15:45:52Z)
• Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
  We present a new learning-based method for multi-frame depth estimation from a color video. Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-02T07:10:45Z)
• Monocular Depth Estimation Based On Deep Learning: An Overview [16.2543991384566]
  Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. Deep learning has been widely studied recently and achieved promising performance in accuracy. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed.
  arXiv  Detail & Related papers  (2020-03-14T12:35:34Z)
• Don't Forget The Past: Recurrent Depth Estimation from Monocular Video [92.84498980104424]
  We put three different types of depth estimation into a common framework. Our method produces a time series of depth maps. It can be applied to monocular videos only or be combined with different types of sparse depth patterns.
  arXiv  Detail & Related papers  (2020-01-08T16:50:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.