MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion

• URL: http://arxiv.org/abs/2504.20040v1
• Date: Mon, 28 Apr 2025 17:59:52 GMT
• Title: MP-SfM: Monocular Surface Priors for Robust Structure-from-Motion
• Authors: Zador Pataki, Paul-Edouard Sarlin, Johannes L. Schönberger, Marc Pollefeys,
• Abstract summary: We present a novel approach to structure-from-Motion (SfM)<n>We show that our approach significantly outperforms existing ones under extreme viewpoint changes.<n>We also show that monocular priors can help reject faulty associations due to symmetries.
• Score: 57.261421172249044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While Structure-from-Motion (SfM) has seen much progress over the years, state-of-the-art systems are prone to failure when facing extreme viewpoint changes in low-overlap, low-parallax or high-symmetry scenarios. Because capturing images that avoid these pitfalls is challenging, this severely limits the wider use of SfM, especially by non-expert users. We overcome these limitations by augmenting the classical SfM paradigm with monocular depth and normal priors inferred by deep neural networks. Thanks to a tight integration of monocular and multi-view constraints, our approach significantly outperforms existing ones under extreme viewpoint changes, while maintaining strong performance in standard conditions. We also show that monocular priors can help reject faulty associations due to symmetries, which is a long-standing problem for SfM. This makes our approach the first capable of reliably reconstructing challenging indoor environments from few images. Through principled uncertainty propagation, it is robust to errors in the priors, can handle priors inferred by different models with little tuning, and will thus easily benefit from future progress in monocular depth and normal estimation. Our code is publicly available at https://github.com/cvg/mpsfm.

Related papers

• Relative Pose Estimation through Affine Corrections of Monocular Depth Priors [69.59216331861437]
  We develop three solvers for relative pose estimation that explicitly account for independent affine (scale and shift) ambiguities.<n>We propose a hybrid estimation pipeline that combines our proposed solvers with classic point-based solvers and epipolar constraints.
  arXiv  Detail & Related papers  (2025-01-09T18:58:30Z)
• Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing [56.66469232740998]
  We show that Structured State Space Models (SSMs) are inherently limited by strong recency bias.<n>This bias impairs the models' ability to recall distant information and introduces robustness issues.<n>We propose to polarize two channels of the state transition matrices in SSMs, setting them to zero and one, respectively, simultaneously addressing recency bias and over-smoothing.
  arXiv  Detail & Related papers  (2024-12-31T22:06:39Z)
• Robust Incremental Structure-from-Motion with Hybrid Features [73.55745864762703]
  We introduce an incremental Structure-from-Motion (SfM) system that leverages lines and their structured geometric relations. Our system is consistently more robust and accurate compared to the widely used point-based state of the art in SfM.
  arXiv  Detail & Related papers  (2024-09-29T22:20:32Z)
• Scene Prior Filtering for Depth Super-Resolution [97.30137398361823]
  We introduce a Scene Prior Filtering network, SPFNet, to mitigate texture interference and edge inaccuracy. Our SPFNet has been extensively evaluated on both real and synthetic datasets, achieving state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-02-21T15:35:59Z)
• DebSDF: Delving into the Details and Bias of Neural Indoor Scene Reconstruction [34.07747722661987]
  This paper focuses on the utilization of uncertainty in monocular priors and the bias in SDF-based volume rendering. We propose an uncertainty modeling technique that associates larger uncertainties with larger errors in the monocular priors. High-uncertainty priors are then excluded from optimization to prevent bias.
  arXiv  Detail & Related papers  (2023-08-29T18:00:22Z)
• Learned Monocular Depth Priors in Visual-Inertial Initialization [4.99761983273316]
  Visual-inertial odometry (VIO) is the pose estimation backbone for most AR/VR and autonomous robotic systems today. We propose to circumvent the limitations of classical visual-inertial structure-from-motion (SfM) We leverage learned monocular depth images (mono-depth) to constrain the relative depth of features, and upgrade the mono-depth to metric scale by jointly optimizing for its scale and shift.
  arXiv  Detail & Related papers  (2022-04-20T00:30:04Z)
• Learning to Estimate Hidden Motions with Global Motion Aggregation [71.12650817490318]
  Occlusions pose a significant challenge to optical flow algorithms that rely on local evidences. We introduce a global motion aggregation module to find long-range dependencies between pixels in the first image. We demonstrate that the optical flow estimates in the occluded regions can be significantly improved without damaging the performance in non-occluded regions.
  arXiv  Detail & Related papers  (2021-04-06T10:32:03Z)
• Blind Image Restoration with Flow Based Priors [19.190289348734215]
  In a blind setting with unknown degradations, a good prior remains crucial. We propose using normalizing flows to model the distribution of the target content and to use this as a prior in a maximum a posteriori (MAP) formulation. To the best of our knowledge, this is the first work that explores normalizing flows as prior in image enhancement problems.
  arXiv  Detail & Related papers  (2020-09-09T21:40:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.