Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimization

• URL: http://arxiv.org/abs/2007.14628v2
• Date: Tue, 8 Sep 2020 02:51:35 GMT
• Title: Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimization
• Authors: Dylan Campbell, Liu Liu, Stephen Gould
• Abstract summary: Blind Perspective-n-Point is the problem estimating the position of a camera relative to a scene. We propose the first fully end-to-end trainable network for solving the blind geometric problem efficiently globally.
• Score: 44.85008070868851
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Blind Perspective-n-Point (PnP) is the problem of estimating the position and orientation of a camera relative to a scene, given 2D image points and 3D scene points, without prior knowledge of the 2D-3D correspondences. Solving for pose and correspondences simultaneously is extremely challenging since the search space is very large. Fortunately it is a coupled problem: the pose can be found easily given the correspondences and vice versa. Existing approaches assume that noisy correspondences are provided, that a good pose prior is available, or that the problem size is small. We instead propose the first fully end-to-end trainable network for solving the blind PnP problem efficiently and globally, that is, without the need for pose priors. We make use of recent results in differentiating optimization problems to incorporate geometric model fitting into an end-to-end learning framework, including Sinkhorn, RANSAC and PnP algorithms. Our proposed approach significantly outperforms other methods on synthetic and real data.

Related papers

• A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose [44.13819148680788]
  We develop a novel construct-and-optimize method for sparse view synthesis without camera poses. Specifically, we construct a solution by using monocular depth and projecting pixels back into the 3D world. We demonstrate results on the Tanks and Temples and Static Hikes datasets with as few as three widely-spaced views.
  arXiv  Detail & Related papers  (2024-05-06T17:36:44Z)
• CheckerPose: Progressive Dense Keypoint Localization for Object Pose Estimation with Graph Neural Network [66.24726878647543]
  Estimating the 6-DoF pose of a rigid object from a single RGB image is a crucial yet challenging task. Recent studies have shown the great potential of dense correspondence-based solutions. We propose a novel pose estimation algorithm named CheckerPose, which improves on three main aspects.
  arXiv  Detail & Related papers  (2023-03-29T17:30:53Z)
• EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation [30.212903535850874]
  Locating 3D objects from a single RGB image via Perspective-n-Point is a long-standing problem in computer vision. EPro-Scene can enhance existing correspondence networks, closing the gap between MOD-based method and the Line 6DoF pose estimation benchmark.
  arXiv  Detail & Related papers  (2023-03-22T17:57:36Z)
• A Solution for a Fundamental Problem of 3D Inference based on 2D Representations [0.0]
  3D inference from monocular vision using neural networks is an important research area of computer vision. This paper provides an explainable and robust-decent solution based on 2D representations for an important special case of the problem. It opens up a new approach for using available information-based learning methods to solve problems related to 3D object pose estimation from 2D images.
  arXiv  Detail & Related papers  (2022-11-09T05:37:01Z)
• Partially calibrated semi-generalized pose from hybrid point correspondences [68.22708881161049]
  We study all possible camera configurations within the generalized camera system. To derive practical solvers, we test different parameterizations as well as different solving strategies. We show that in the presence of noise in the 3D points these solvers provide better estimates than the corresponding absolute pose solvers.
  arXiv  Detail & Related papers  (2022-09-29T19:46:59Z)
• Coupled Iterative Refinement for 6D Multi-Object Pose Estimation [64.7198752089041]
  Given a set of known 3D objects and an RGB or RGB-D input image, we detect and estimate the 6D pose of each object. Our approach iteratively refines both pose and correspondence in a tightly coupled manner, allowing us to dynamically remove outliers to improve accuracy.
  arXiv  Detail & Related papers  (2022-04-26T18:00:08Z)
• Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimation [74.76155168705975]
  Deep Bingham Networks (DBN) can handle pose-related uncertainties and ambiguities arising in almost all real life applications concerning 3D data. DBN extends the state of the art direct pose regression networks by (i) a multi-hypotheses prediction head which can yield different distribution modes. We propose new training strategies so as to avoid mode or posterior collapse during training and to improve numerical stability.
  arXiv  Detail & Related papers  (2020-12-20T19:20:26Z)
• Learning 2D-3D Correspondences To Solve The Blind Perspective-n-Point Problem [98.92148855291363]
  This paper proposes a deep CNN model which simultaneously solves for both 6-DoF absolute camera pose 2D--3D correspondences. Tests on both real and simulated data have shown that our method substantially outperforms existing approaches.
  arXiv  Detail & Related papers  (2020-03-15T04:17:30Z)
• PnP-Net: A hybrid Perspective-n-Point Network [2.66512000865131]
  We consider the robust Perspective-n-Point problem using a hybrid approach that combines deep learning with model based algorithms. We demonstrate both synthetic parameters and real world data with low computational requirements.
  arXiv  Detail & Related papers  (2020-03-10T10:43:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.