Map-Relative Pose Regression for Visual Re-Localization

• URL: http://arxiv.org/abs/2404.09884v1
• Date: Mon, 15 Apr 2024 15:53:23 GMT
• Title: Map-Relative Pose Regression for Visual Re-Localization
• Authors: Shuai Chen, Tommaso Cavallari, Victor Adrian Prisacariu, Eric Brachmann,
• Abstract summary: We present a new approach to pose regression, map-relative pose regression (marepo) We condition the pose regressor on a scene-specific map representation such that its pose predictions are relative to the scene map. Our approach outperforms previous pose regression methods by far on two public datasets, indoor and outdoor.
• Score: 20.89982939633994
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pose regression networks predict the camera pose of a query image relative to a known environment. Within this family of methods, absolute pose regression (APR) has recently shown promising accuracy in the range of a few centimeters in position error. APR networks encode the scene geometry implicitly in their weights. To achieve high accuracy, they require vast amounts of training data that, realistically, can only be created using novel view synthesis in a days-long process. This process has to be repeated for each new scene again and again. We present a new approach to pose regression, map-relative pose regression (marepo), that satisfies the data hunger of the pose regression network in a scene-agnostic fashion. We condition the pose regressor on a scene-specific map representation such that its pose predictions are relative to the scene map. This allows us to train the pose regressor across hundreds of scenes to learn the generic relation between a scene-specific map representation and the camera pose. Our map-relative pose regressor can be applied to new map representations immediately or after mere minutes of fine-tuning for the highest accuracy. Our approach outperforms previous pose regression methods by far on two public datasets, indoor and outdoor. Code is available: https://nianticlabs.github.io/marepo

Related papers

• PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching [51.142988196855484]
  We propose PoseMatcher, an accurate model free one-shot object pose estimator. We create a new training pipeline for object to image matching based on a three-view system. To enable PoseMatcher to attend to distinct input modalities, an image and a pointcloud, we introduce IO-Layer.
  arXiv  Detail & Related papers  (2023-04-03T21:14:59Z)
• A Probabilistic Framework for Visual Localization in Ambiguous Scenes [64.13544430239267]
  We propose a probabilistic framework that for a given image predicts the arbitrarily shaped posterior distribution of its camera pose. We do this via a novel formulation of camera pose regression using variational inference, which allows sampling from the predicted distribution. Our method outperforms existing methods on localization in ambiguous scenes.
  arXiv  Detail & Related papers  (2023-01-05T14:46:54Z)
• Camera Pose Auto-Encoders for Improving Pose Regression [6.700873164609009]
  We introduce Camera Pose Auto-Encoders (PAEs) to encode camera poses using APRs as their teachers. We show that the resulting latent pose representations can closely reproduce APR performance and demonstrate their effectiveness for related tasks. We also show that train images can be reconstructed from the learned pose encoding, paving the way for integrating visual information from the train set at a low memory cost.
  arXiv  Detail & Related papers  (2022-07-12T13:47:36Z)
• Subpixel Heatmap Regression for Facial Landmark Localization [65.41270740933656]
  Heatmap regression approaches suffer from discretization-induced errors related to both the heatmap encoding and decoding process. We propose a new approach for the heatmap encoding and decoding process by leveraging the underlying continuous distribution. Our approach offers noticeable gains across multiple datasets setting a new state-of-the-art result in facial landmark localization.
  arXiv  Detail & Related papers  (2021-11-03T17:21:28Z)
• Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision [31.947525258453584]
  Visual re-localization means using a single image as input to estimate the camera's location and orientation relative to a pre-recorded environment. Our proposed method makes few special assumptions, and is fairly lightweight in training and testing. We validate the effectiveness of our approach on both standard indoor (7-Scenes) and outdoor (Cambridge Landmarks) camera re-localization benchmarks.
  arXiv  Detail & Related papers  (2021-04-06T14:29:03Z)
• Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression [81.05772887221333]
  We study the dense keypoint regression framework that is previously inferior to the keypoint detection and grouping framework. We present a simple yet effective approach, named disentangled keypoint regression (DEKR) We empirically show that the proposed direct regression method outperforms keypoint detection and grouping methods.
  arXiv  Detail & Related papers  (2021-04-06T05:54:46Z)
• Paying Attention to Activation Maps in Camera Pose Regression [4.232614032390374]
  Camera pose regression methods apply a single forward pass to the query image to estimate the camera pose. We propose an attention-based approach for pose regression, where the convolutional activation maps are used as sequential inputs. Our proposed approach is shown to compare favorably to contemporary pose regressors schemes and achieves state-of-the-art accuracy across multiple benchmarks.
  arXiv  Detail & Related papers  (2021-03-21T20:10:15Z)
• Back to the Feature: Learning Robust Camera Localization from Pixels to Pose [114.89389528198738]
  We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching.
  arXiv  Detail & Related papers  (2021-03-16T17:40:12Z)
• Do We Really Need Scene-specific Pose Encoders? [0.0]
  Visual pose regression models estimate the camera pose from a query image with a single forward pass. Current models learn pose encoding from an image using deep convolutional networks which are trained per scene. We propose that scene-specific pose encoders are not required for pose regression and that encodings trained for visual similarity can be used instead.
  arXiv  Detail & Related papers  (2020-12-22T13:59:52Z)
• Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates [76.51095823248104]
  We present several schemes that are rarely or unthoroughly studied before for improving keypoint detection and grouping (keypoint regression) performance. First, we exploit the keypoint heatmaps for pixel-wise keypoint regression instead of separating them for improving keypoint regression. Second, we adopt a pixel-wise spatial transformer network to learn adaptive representations for handling the scale and orientation variance. Third, we present a joint shape and heatvalue scoring scheme to promote the estimated poses that are more likely to be true poses.
  arXiv  Detail & Related papers  (2020-06-28T01:14:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.