Multi-view Reconstruction via SfM-guided Monocular Depth Estimation

• URL: http://arxiv.org/abs/2503.14483v1
• Date: Tue, 18 Mar 2025 17:54:06 GMT
• Title: Multi-view Reconstruction via SfM-guided Monocular Depth Estimation
• Authors: Haoyu Guo, He Zhu, Sida Peng, Haotong Lin, Yunzhi Yan, Tao Xie, Wenguan Wang, Xiaowei Zhou, Hujun Bao,
• Abstract summary: We present a new method for multi-view geometric reconstruction.<n>We incorporate SfM information, a strong multi-view prior, into the depth estimation process.<n>Our method significantly improves the quality of depth estimation compared to previous monocular depth estimation works.
• Score: 92.89227629434316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we present a new method for multi-view geometric reconstruction. In recent years, large vision models have rapidly developed, performing excellently across various tasks and demonstrating remarkable generalization capabilities. Some works use large vision models for monocular depth estimation, which have been applied to facilitate multi-view reconstruction tasks in an indirect manner. Due to the ambiguity of the monocular depth estimation task, the estimated depth values are usually not accurate enough, limiting their utility in aiding multi-view reconstruction. We propose to incorporate SfM information, a strong multi-view prior, into the depth estimation process, thus enhancing the quality of depth prediction and enabling their direct application in multi-view geometric reconstruction. Experimental results on public real-world datasets show that our method significantly improves the quality of depth estimation compared to previous monocular depth estimation works. Additionally, we evaluate the reconstruction quality of our approach in various types of scenes including indoor, streetscape, and aerial views, surpassing state-of-the-art MVS methods. The code and supplementary materials are available at https://zju3dv.github.io/murre/ .

Related papers

• Align3R: Aligned Monocular Depth Estimation for Dynamic Videos [50.28715151619659]
  We propose a novel video-depth estimation method called Align3R to estimate temporal consistent depth maps for a dynamic video. Our key idea is to utilize the recent DUSt3R model to align estimated monocular depth maps of different timesteps. Experiments demonstrate that Align3R estimates consistent video depth and camera poses for a monocular video with superior performance than baseline methods.
  arXiv  Detail & Related papers  (2024-12-04T07:09:59Z)
• Robust Geometry-Preserving Depth Estimation Using Differentiable Rendering [93.94371335579321]
  We propose a learning framework that trains models to predict geometry-preserving depth without requiring extra data or annotations. Comprehensive experiments underscore our framework's superior generalization capabilities. Our innovative loss functions empower the model to autonomously recover domain-specific scale-and-shift coefficients.
  arXiv  Detail & Related papers  (2023-09-18T12:36:39Z)
• Incremental Dense Reconstruction from Monocular Video with Guided Sparse Feature Volume Fusion [23.984073189849024]
  This letter proposes a real-time feature volume-based dense reconstruction method that predicts TSDF values from a novel sparsified deep feature volume. An uncertainty-aware multi-view stereo network is leveraged to infer initial voxel locations of the physical surface in a sparse feature volume. Our method is shown to produce more complete reconstructions with finer detail in many cases.
  arXiv  Detail & Related papers  (2023-05-24T09:06:01Z)
• FusionDepth: Complement Self-Supervised Monocular Depth Estimation with Cost Volume [9.912304015239313]
  We propose a multi-frame depth estimation framework which monocular depth can be refined continuously by multi-frame sequential constraints. Our method also enhances the interpretability when combining monocular estimation with multi-view cost volume.
  arXiv  Detail & Related papers  (2023-05-10T10:38:38Z)
• Towards Domain-agnostic Depth Completion [28.25756709062647]
  Existing depth completion methods are often targeted at a specific sparse depth type and generalize poorly across task domains. We present a method to complete sparse/semi-dense, noisy, and potentially low-resolution depth maps obtained by various range sensors. Our method shows superior cross-domain generalization ability against state-of-the-art depth completion methods.
  arXiv  Detail & Related papers  (2022-07-29T04:10:22Z)
• MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction [72.05649682685197]
  State-of-the-art neural implicit methods allow for high-quality reconstructions of simple scenes from many input views. This is caused primarily by the inherent ambiguity in the RGB reconstruction loss that does not provide enough constraints. Motivated by recent advances in the area of monocular geometry prediction, we explore the utility these cues provide for improving neural implicit surface reconstruction.
  arXiv  Detail & Related papers  (2022-06-01T17:58:15Z)
• Improving Monocular Visual Odometry Using Learned Depth [84.05081552443693]
  We propose a framework to exploit monocular depth estimation for improving visual odometry (VO) The core of our framework is a monocular depth estimation module with a strong generalization capability for diverse scenes. Compared with current learning-based VO methods, our method demonstrates a stronger generalization ability to diverse scenes.
  arXiv  Detail & Related papers  (2022-04-04T06:26:46Z)
• DDL-MVS: Depth Discontinuity Learning for MVS Networks [0.5735035463793007]
  We propose depth discontinuity learning for MVS methods, which further improves accuracy while retaining the completeness of the reconstruction. We validate our idea and demonstrate that our strategies can be easily integrated into the existing learning-based MVS pipeline.
  arXiv  Detail & Related papers  (2022-03-02T20:25:31Z)
• Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks [87.50632573601283]
  We present a novel method for multi-view depth estimation from a single video. Our method achieves temporally coherent depth estimation results by using a novel Epipolar Spatio-Temporal (EST) transformer. To reduce the computational cost, inspired by recent Mixture-of-Experts models, we design a compact hybrid network.
  arXiv  Detail & Related papers  (2020-11-26T04:04:21Z)
• Monocular Depth Estimation Based On Deep Learning: An Overview [16.2543991384566]
  Inferring depth information from a single image (monocular depth estimation) is an ill-posed problem. Deep learning has been widely studied recently and achieved promising performance in accuracy. In order to improve the accuracy of depth estimation, different kinds of network frameworks, loss functions and training strategies are proposed.
  arXiv  Detail & Related papers  (2020-03-14T12:35:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.