Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior

• URL: http://arxiv.org/abs/2502.06338v1
• Date: Mon, 10 Feb 2025 10:38:33 GMT
• Title: Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior
• Authors: Lee Hyoseok, Kyeong Seon Kim, Kwon Byung-Ki, Tae-Hyun Oh,
• Abstract summary: We propose a zero-shot depth completion method composed of an affine-invariant depth diffusion model and test-time alignment. Our approach aligns the affine-invariant depth prior with metric-scale sparse measurements, enforcing them as hard constraints via an optimization loop at test-time.
• Score: 15.802986215292309
• License:
• Abstract: Depth completion, predicting dense depth maps from sparse depth measurements, is an ill-posed problem requiring prior knowledge. Recent methods adopt learning-based approaches to implicitly capture priors, but the priors primarily fit in-domain data and do not generalize well to out-of-domain scenarios. To address this, we propose a zero-shot depth completion method composed of an affine-invariant depth diffusion model and test-time alignment. We use pre-trained depth diffusion models as depth prior knowledge, which implicitly understand how to fill in depth for scenes. Our approach aligns the affine-invariant depth prior with metric-scale sparse measurements, enforcing them as hard constraints via an optimization loop at test-time. Our zero-shot depth completion method demonstrates generalization across various domain datasets, achieving up to a 21\% average performance improvement over the previous state-of-the-art methods while enhancing spatial understanding by sharpening scene details. We demonstrate that aligning a monocular affine-invariant depth prior with sparse metric measurements is a proven strategy to achieve domain-generalizable depth completion without relying on extensive training data. Project page: https://hyoseok1223.github.io/zero-shot-depth-completion/.

Related papers

• Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion [51.69876947593144]
  Existing methods for depth completion operate in tightly constrained settings. Inspired by advances in monocular depth estimation, we reframe depth completion as an image-conditional depth map generation. Marigold-DC builds on a pretrained latent diffusion model for monocular depth estimation and injects the depth observations as test-time guidance.
  arXiv  Detail & Related papers  (2024-12-18T00:06:41Z)
• ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation [62.600382533322325]
  We propose a novel monocular depth estimation method called ScaleDepth. Our method decomposes metric depth into scene scale and relative depth, and predicts them through a semantic-aware scale prediction module. Our method achieves metric depth estimation for both indoor and outdoor scenes in a unified framework.
  arXiv  Detail & Related papers  (2024-07-11T05:11:56Z)
• Temporal Lidar Depth Completion [0.08192907805418582]
  We show how a state-of-the-art method PENet can be modified to benefit from recurrency. Our algorithm achieves state-of-the-art results on the KITTI depth completion dataset.
  arXiv  Detail & Related papers  (2024-06-17T08:25:31Z)
• Depth Prompting for Sensor-Agnostic Depth Estimation [19.280536006736575]
  We design a novel depth prompt module to allow the desirable feature representation according to new depth distributions. Our method helps the pretrained model to be free from restraint of depth scan range and to provide absolute scale depth maps.
  arXiv  Detail & Related papers  (2024-05-20T08:19:08Z)
• Towards Domain-agnostic Depth Completion [28.25756709062647]
  Existing depth completion methods are often targeted at a specific sparse depth type and generalize poorly across task domains. We present a method to complete sparse/semi-dense, noisy, and potentially low-resolution depth maps obtained by various range sensors. Our method shows superior cross-domain generalization ability against state-of-the-art depth completion methods.
  arXiv  Detail & Related papers  (2022-07-29T04:10:22Z)
• Efficient Depth Completion Using Learned Bases [94.0808155168311]
  We propose a new global geometry constraint for depth completion. By assuming depth maps often lay on low dimensional subspaces, a dense depth map can be approximated by a weighted sum of full-resolution principal depth bases.
  arXiv  Detail & Related papers  (2020-12-02T11:57:37Z)
• Occlusion-Aware Depth Estimation with Adaptive Normal Constraints [85.44842683936471]
  We present a new learning-based method for multi-frame depth estimation from a color video. Our method outperforms the state-of-the-art in terms of depth estimation accuracy.
  arXiv  Detail & Related papers  (2020-04-02T07:10:45Z)
• DiverseDepth: Affine-invariant Depth Prediction Using Diverse Data [110.29043712400912]
  We present a method for depth estimation with monocular images, which can predict high-quality depth on diverse scenes up to an affine transformation. Experiments show that our method outperforms previous methods on 8 datasets by a large margin with the zero-shot test setting.
  arXiv  Detail & Related papers  (2020-02-03T05:38:33Z)
• Don't Forget The Past: Recurrent Depth Estimation from Monocular Video [92.84498980104424]
  We put three different types of depth estimation into a common framework. Our method produces a time series of depth maps. It can be applied to monocular videos only or be combined with different types of sparse depth patterns.
  arXiv  Detail & Related papers  (2020-01-08T16:50:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.