The Devil is in the Edges: Monocular Depth Estimation with Edge-aware Consistency Fusion

• URL: http://arxiv.org/abs/2404.00373v1
• Date: Sat, 30 Mar 2024 13:58:19 GMT
• Title: The Devil is in the Edges: Monocular Depth Estimation with Edge-aware Consistency Fusion
• Authors: Pengzhi Li, Yikang Ding, Haohan Wang, Chengshuai Tang, Zhiheng Li,
• Abstract summary: This paper presents a novel monocular depth estimation method, named ECFNet, for estimating high-quality monocular depth with clear edges and valid overall structure from a single RGB image. We make a thorough inquiry about the key factor that affects the edge depth estimation of the MDE networks, and come to a ratiocination that the edge information itself plays a critical role in predicting depth details.
• Score: 30.03608191629917
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a novel monocular depth estimation method, named ECFNet, for estimating high-quality monocular depth with clear edges and valid overall structure from a single RGB image. We make a thorough inquiry about the key factor that affects the edge depth estimation of the MDE networks, and come to a ratiocination that the edge information itself plays a critical role in predicting depth details. Driven by this analysis, we propose to explicitly employ the image edges as input for ECFNet and fuse the initial depths from different sources to produce the final depth. Specifically, ECFNet first uses a hybrid edge detection strategy to get the edge map and edge-highlighted image from the input image, and then leverages a pre-trained MDE network to infer the initial depths of the aforementioned three images. After that, ECFNet utilizes a layered fusion module (LFM) to fuse the initial depth, which will be further updated by a depth consistency module (DCM) to form the final estimation. Extensive experimental results on public datasets and ablation studies indicate that our method achieves state-of-the-art performance. Project page: https://zrealli.github.io/edgedepth.

Related papers

• Depth Estimation Based on 3D Gaussian Splatting Siamese Defocus [14.354405484663285]
  We propose a self-supervised framework based on 3D Gaussian splatting and Siamese networks for depth estimation in 3D geometry. The proposed framework has been validated on both artificially synthesized and real blurred datasets.
  arXiv  Detail & Related papers  (2024-09-18T21:36:37Z)
• Robust Depth Enhancement via Polarization Prompt Fusion Tuning [112.88371907047396]
  We present a framework that leverages polarization imaging to improve inaccurate depth measurements from various depth sensors. Our method first adopts a learning-based strategy where a neural network is trained to estimate a dense and complete depth map from polarization data and a sensor depth map from different sensors. To further improve the performance, we propose a Polarization Prompt Fusion Tuning (PPFT) strategy to effectively utilize RGB-based models pre-trained on large-scale datasets.
  arXiv  Detail & Related papers  (2024-04-05T17:55:33Z)
• Bilateral Propagation Network for Depth Completion [41.163328523175466]
  Depth completion aims to derive a dense depth map from sparse depth measurements with a synchronized color image. Current state-of-the-art (SOTA) methods are predominantly propagation-based, which work as an iterative refinement on the initial estimated dense depth. We present a Bilateral Propagation Network (BP-Net), that propagates depth at the earliest stage to avoid directly convolving on sparse data.
  arXiv  Detail & Related papers  (2024-03-17T16:48:46Z)
• Monocular Depth Estimation Primed by Salient Point Detection and Normalized Hessian Loss [43.950140695759764]
  We propose an accurate and lightweight framework for monocular depth estimation based on a self-attention mechanism stemming from salient point detection. We introduce a normalized Hessian loss term invariant to scaling and shear along the depth direction, which is shown to substantially improve the accuracy. The proposed method achieves state-of-the-art results on NYU-Depth-v2 and KITTI while using 3.1-38.4 times smaller model in terms of the number of parameters than baseline approaches.
  arXiv  Detail & Related papers  (2021-08-25T07:51:09Z)
• Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion [56.85837052421469]
  Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars. In this paper, we study the problem of predicting dense depth from a single RGB image with optional sparse measurements from low-cost active depth sensors. We introduce Sparse Networks (SANs), a new module enabling monodepth networks to perform both the tasks of depth prediction and completion.
  arXiv  Detail & Related papers  (2021-03-30T21:22:26Z)
• Boundary-induced and scene-aggregated network for monocular depth prediction [20.358133522462513]
  We propose the Boundary-induced and Scene-aggregated network (BS-Net) to predict the dense depth of a single RGB image. Several experimental results on the NYUD v2 dataset and xffthe iBims-1 dataset illustrate the state-of-the-art performance of the proposed approach.
  arXiv  Detail & Related papers  (2021-02-26T01:43:17Z)
• Deep Multi-Scale Feature Learning for Defocus Blur Estimation [10.455763145066168]
  This paper presents an edge-based defocus blur estimation method from a single defocused image. We first distinguish edges that lie at depth discontinuities (called depth edges, for which the blur estimate is ambiguous) from edges that lie at approximately constant depth regions (called pattern edges, for which the blur estimate is well-defined). We estimate the defocus blur amount at pattern edges only, and explore an scheme based on guided filters that prevents data propagation across the detected depth edges to obtain a dense blur map with well-defined object boundaries.
  arXiv  Detail & Related papers  (2020-09-24T20:36:40Z)
• A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection [89.88222217065858]
  We design a single stream network to use the depth map to guide early fusion and middle fusion between RGB and depth. This model is 55.5% lighter than the current lightest model and runs at a real-time speed of 32 FPS when processing a $384 times 384$ image.
  arXiv  Detail & Related papers  (2020-07-14T04:40:14Z)
• Depth Completion Using a View-constrained Deep Prior [73.21559000917554]
  Recent work has shown that the structure of convolutional neural networks (CNNs) induces a strong prior that favors natural images. This prior, known as a deep image prior (DIP), is an effective regularizer in inverse problems such as image denoising and inpainting. We extend the concept of the DIP to depth images. Given color images and noisy and incomplete target depth maps, we reconstruct a depth map restored by virtue of using the CNN network structure as a prior.
  arXiv  Detail & Related papers  (2020-01-21T21:56:01Z)
• Single Image Depth Estimation Trained via Depth from Defocus Cues [105.67073923825842]
  Estimating depth from a single RGB image is a fundamental task in computer vision. In this work, we rely, instead of different views, on depth from focus cues. We present results that are on par with supervised methods on KITTI and Make3D datasets and outperform unsupervised learning approaches.
  arXiv  Detail & Related papers  (2020-01-14T20:22:54Z)
• Video Depth Estimation by Fusing Flow-to-Depth Proposals [65.24533384679657]
  We present an approach with a differentiable flow-to-depth layer for video depth estimation. The model consists of a flow-to-depth layer, a camera pose refinement module, and a depth fusion network. Our approach outperforms state-of-the-art depth estimation methods, and has reasonable cross dataset generalization capability.
  arXiv  Detail & Related papers  (2019-12-30T10:45:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.