Relative Pose Estimation through Affine Corrections of Monocular Depth Priors

• URL: http://arxiv.org/abs/2501.05446v1
• Date: Thu, 09 Jan 2025 18:58:30 GMT
• Title: Relative Pose Estimation through Affine Corrections of Monocular Depth Priors
• Authors: Yifan Yu, Shaohui Liu, Rémi Pautrat, Marc Pollefeys, Viktor Larsson,
• Abstract summary: We develop three solvers for relative pose estimation that explicitly account for independent affine (scale and shift) ambiguities. We propose a hybrid estimation pipeline that combines our proposed solvers with classic point-based solvers and epipolar constraints.
• Score: 69.59216331861437
• License:
• Abstract: Monocular depth estimation (MDE) models have undergone significant advancements over recent years. Many MDE models aim to predict affine-invariant relative depth from monocular images, while recent developments in large-scale training and vision foundation models enable reasonable estimation of metric (absolute) depth. However, effectively leveraging these predictions for geometric vision tasks, in particular relative pose estimation, remains relatively under explored. While depths provide rich constraints for cross-view image alignment, the intrinsic noise and ambiguity from the monocular depth priors present practical challenges to improving upon classic keypoint-based solutions. In this paper, we develop three solvers for relative pose estimation that explicitly account for independent affine (scale and shift) ambiguities, covering both calibrated and uncalibrated conditions. We further propose a hybrid estimation pipeline that combines our proposed solvers with classic point-based solvers and epipolar constraints. We find that the affine correction modeling is beneficial to not only the relative depth priors but also, surprisingly, the ``metric" ones. Results across multiple datasets demonstrate large improvements of our approach over classic keypoint-based baselines and PnP-based solutions, under both calibrated and uncalibrated setups. We also show that our method improves consistently with different feature matchers and MDE models, and can further benefit from very recent advances on both modules. Code is available at https://github.com/MarkYu98/madpose.

Related papers

• UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image [86.7128543480229]
  We present a novel approach and benchmark, termed UNOPose, for unseen one-reference-based object pose estimation. Building upon a coarse-to-fine paradigm, UNOPose constructs an SE(3)-invariant reference frame to standardize object representation. We recalibrate the weight of each correspondence based on its predicted likelihood of being within the overlapping region.
  arXiv  Detail & Related papers  (2024-11-25T05:36:00Z)
• Consistency Regularisation for Unsupervised Domain Adaptation in Monocular Depth Estimation [15.285720572043678]
  We formulate unsupervised domain adaptation for monocular depth estimation as a consistency-based semi-supervised learning problem. We introduce a pairwise loss function that regularises predictions on the source domain while enforcing consistency across multiple augmented views. In our experiments, we rely on the standard depth estimation benchmarks KITTI and NYUv2 to demonstrate state-of-the-art results.
  arXiv  Detail & Related papers  (2024-05-27T23:32:06Z)
• IEBins: Iterative Elastic Bins for Monocular Depth Estimation [25.71386321706134]
  We propose a novel concept of iterative elastic bins (IEBins) for the classification-regression-based MDE. The proposed IEBins aims to search for high-quality depth by progressively optimizing the search range. We develop a dedicated framework composed of a feature extractor and an iterative framework benefiting from the GRU-based architecture.
  arXiv  Detail & Related papers  (2023-09-25T13:48:39Z)
• FrozenRecon: Pose-free 3D Scene Reconstruction with Frozen Depth Models [67.96827539201071]
  We propose a novel test-time optimization approach for 3D scene reconstruction. Our method achieves state-of-the-art cross-dataset reconstruction on five zero-shot testing datasets.
  arXiv  Detail & Related papers  (2023-08-10T17:55:02Z)
• DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium [11.78276690882616]
  Self-supervised multi-frame depth estimation achieves high accuracy by computing matching costs of pixel correspondences between adjacent frames. We propose the Dual model, which tightly couples depth and pose estimation through a feedback loop. Our novel update pipeline uses a deep equilibrium model framework to iteratively refine depth estimates and a hidden state of feature maps.
  arXiv  Detail & Related papers  (2023-04-07T09:46:29Z)
• Single Image Depth Prediction Made Better: A Multivariate Gaussian Take [163.14849753700682]
  We introduce an approach that performs continuous modeling of per-pixel depth. Our method's accuracy (named MG) is among the top on the KITTI depth-prediction benchmark leaderboard.
  arXiv  Detail & Related papers  (2023-03-31T16:01:03Z)
• DeepMLE: A Robust Deep Maximum Likelihood Estimator for Two-view Structure from Motion [9.294501649791016]
  Two-view structure from motion (SfM) is the cornerstone of 3D reconstruction and visual SLAM (vSLAM) We formulate the two-view SfM problem as a maximum likelihood estimation (MLE) and solve it with the proposed framework, denoted as DeepMLE. Our method significantly outperforms the state-of-the-art end-to-end two-view SfM approaches in accuracy and generalization capability.
  arXiv  Detail & Related papers  (2022-10-11T15:07:25Z)
• Exploiting Correspondences with All-pairs Correlations for Multi-view Depth Estimation [19.647670347925754]
  Multi-view depth estimation plays a critical role in reconstructing and understanding the 3D world. We design a novel iterative multi-view depth estimation framework mimicking the optimization process. We conduct sufficient experiments on ScanNet, DeMoN, ETH3D, and 7Scenes to demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-05-05T07:38:31Z)
• A Model for Multi-View Residual Covariances based on Perspective Deformation [88.21738020902411]
  We derive a model for the covariance of the visual residuals in multi-view SfM, odometry and SLAM setups. We validate our model with synthetic and real data and integrate it into photometric and feature-based Bundle Adjustment.
  arXiv  Detail & Related papers  (2022-02-01T21:21:56Z)
• Deep Keypoint-Based Camera Pose Estimation with Geometric Constraints [80.60538408386016]
  Estimating relative camera poses from consecutive frames is a fundamental problem in visual odometry. We propose an end-to-end trainable framework consisting of learnable modules for detection, feature extraction, matching and outlier rejection.
  arXiv  Detail & Related papers  (2020-07-29T21:41:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.