DDL-MVS: Depth Discontinuity Learning for MVS Networks

• URL: http://arxiv.org/abs/2203.01391v3
• Date: Mon, 12 Jun 2023 12:05:42 GMT
• Title: DDL-MVS: Depth Discontinuity Learning for MVS Networks
• Authors: Nail Ibrahimli, Hugo Ledoux, Julian Kooij, Liangliang Nan
• Abstract summary: We propose depth discontinuity learning for MVS methods, which further improves accuracy while retaining the completeness of the reconstruction. We validate our idea and demonstrate that our strategies can be easily integrated into the existing learning-based MVS pipeline.
• Score: 0.5735035463793007
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Traditional MVS methods have good accuracy but struggle with completeness, while recently developed learning-based multi-view stereo (MVS) techniques have improved completeness except accuracy being compromised. We propose depth discontinuity learning for MVS methods, which further improves accuracy while retaining the completeness of the reconstruction. Our idea is to jointly estimate the depth and boundary maps where the boundary maps are explicitly used for further refinement of the depth maps. We validate our idea and demonstrate that our strategies can be easily integrated into the existing learning-based MVS pipeline where the reconstruction depends on high-quality depth map estimation. Extensive experiments on various datasets show that our method improves reconstruction quality compared to baseline. Experiments also demonstrate that the presented model and strategies have good generalization capabilities. The source code will be available soon.

Related papers

• Learning-based Multi-View Stereo: A Survey [55.3096230732874]
  Multi-View Stereo (MVS) algorithms synthesize a comprehensive 3D representation, enabling precise reconstruction in complex environments. With the success of deep learning, many learning-based MVS methods have been proposed, achieving impressive performance against traditional methods.
  arXiv  Detail & Related papers  (2024-08-27T17:53:18Z)
• V-FUSE: Volumetric Depth Map Fusion with Long-Range Constraints [6.7197802356130465]
  We introduce a learning-based depth map fusion framework that accepts a set of depth and confidence maps generated by a Multi-View Stereo (MVS) algorithm as input and improves them. We also introduce a depth search window estimation sub-network trained jointly with the larger fusion sub-network to reduce the depth hypothesis search space along each ray. Our method learns to model depth consensus and violations of visibility constraints directly from the data.
  arXiv  Detail & Related papers  (2023-08-17T00:39:56Z)
• Sparse Depth-Guided Attention for Accurate Depth Completion: A Stereo-Assisted Monitored Distillation Approach [7.902840502973506]
  We introduce a stereo-based model as a teacher model to improve the accuracy of the student model for depth completion. To provide self-supervised information, we also employ multi-view depth consistency and multi-scale minimum reprojection.
  arXiv  Detail & Related papers  (2023-03-28T09:23:19Z)
• Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning [22.828829870704006]
  Self-supervised monocular methods can efficiently learn depth information of weakly textured surfaces or reflective objects. In contrast, multi-frame depth estimation methods improve the depth accuracy thanks to the success of Multi-View Stereo. We propose MOVEDepth, which exploits the MOnocular cues and VE guidance to improve multi-frame depth learning.
  arXiv  Detail & Related papers  (2022-08-19T06:32:06Z)
• DS-MVSNet: Unsupervised Multi-view Stereo via Depth Synthesis [11.346448410152844]
  In this paper, we propose the DS-MVSNet, an end-to-end unsupervised MVS structure with the source depths synthesis. To mine the information in probability volume, we creatively synthesize the source depths by splattering the probability volume and depth hypotheses to source views. On the other hand, we utilize the source depths to render the reference images and propose depth consistency loss and depth smoothness loss.
  arXiv  Detail & Related papers  (2022-08-13T15:25:51Z)
• Improving Monocular Visual Odometry Using Learned Depth [84.05081552443693]
  We propose a framework to exploit monocular depth estimation for improving visual odometry (VO) The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes. Compared with current learning-based VO methods, our method demonstrates a stronger generalization ability to diverse scenes.
  arXiv  Detail & Related papers  (2022-04-04T06:26:46Z)
• 3DVNet: Multi-View Depth Prediction and Volumetric Refinement [68.68537312256144]
  3DVNet is a novel multi-view stereo (MVS) depth-prediction method. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions. We show that our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics.
  arXiv  Detail & Related papers  (2021-12-01T00:52:42Z)
• VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction [71.83308989022635]
  In this paper, we advocate that replicating the traditional two stages framework with deep neural networks improves both the interpretability and the accuracy of the results. Our network operates in two steps: 1) the local computation of the local depth maps with a deep MVS technique, and, 2) the depth maps and images' features fusion to build a single TSDF volume. In order to improve the matching performance between images acquired from very different viewpoints, we introduce a rotation-invariant 3D convolution kernel called PosedConv.
  arXiv  Detail & Related papers  (2021-08-19T11:33:58Z)
• Towards Unpaired Depth Enhancement and Super-Resolution in the Wild [121.96527719530305]
  State-of-the-art data-driven methods of depth map super-resolution rely on registered pairs of low- and high-resolution depth maps of the same scenes. We consider an approach to depth map enhancement based on learning from unpaired data.
  arXiv  Detail & Related papers  (2021-05-25T16:19:16Z)
• Attention Aware Cost Volume Pyramid Based Multi-view Stereo Network for 3D Reconstruction [12.728154351588053]
  We present an efficient multi-view stereo (MVS) network for 3D reconstruction from multiview images. We introduce a coarseto-fine depth inference strategy to achieve high resolution depth.
  arXiv  Detail & Related papers  (2020-11-25T13:34:11Z)
• Channel Attention based Iterative Residual Learning for Depth Map Super-Resolution [58.626803922196146]
  We argue that DSR models trained on synthetic dataset are restrictive and not effective in dealing with real-world DSR tasks. We make two contributions in tackling real-world degradation of different depth sensors. We propose a new framework for real-world DSR, which consists of four modules.
  arXiv  Detail & Related papers  (2020-06-02T09:12:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.