Self-Supervised Feature Learning for Long-Term Metric Visual Localization

• URL: http://arxiv.org/abs/2212.00122v1
• Date: Wed, 30 Nov 2022 21:15:05 GMT
• Title: Self-Supervised Feature Learning for Long-Term Metric Visual Localization
• Authors: Yuxuan Chen, Timothy D. Barfoot
• Abstract summary: We present a novel self-supervised feature learning framework for metric visual localization. We use a sequence-based image matching algorithm to generate image correspondences without ground-truth labels. We can then sample image pairs to train a deep neural network that learns sparse features with associated descriptors and scores without ground-truth pose supervision.
• Score: 16.987148593917905
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual localization is the task of estimating camera pose in a known scene, which is an essential problem in robotics and computer vision. However, long-term visual localization is still a challenge due to the environmental appearance changes caused by lighting and seasons. While techniques exist to address appearance changes using neural networks, these methods typically require ground-truth pose information to generate accurate image correspondences or act as a supervisory signal during training. In this paper, we present a novel self-supervised feature learning framework for metric visual localization. We use a sequence-based image matching algorithm across different sequences of images (i.e., experiences) to generate image correspondences without ground-truth labels. We can then sample image pairs to train a deep neural network that learns sparse features with associated descriptors and scores without ground-truth pose supervision. The learned features can be used together with a classical pose estimator for visual stereo localization. We validate the learned features by integrating with an existing Visual Teach & Repeat pipeline to perform closed-loop localization experiments under different lighting conditions for a total of 22.4 km.

Related papers

• Exploring Emerging Trends and Research Opportunities in Visual Place Recognition [28.76562316749074]
  Visual-based recognition is a long-standing challenge in computer vision and robotics communities. Visual place recognition is vital for most localization implementations. Researchers have recently turned their attention to vision-language models.
  arXiv  Detail & Related papers  (2024-11-18T11:36:17Z)
• Self-supervised Learning of Neural Implicit Feature Fields for Camera Pose Refinement [32.335953514942474]
  This paper proposes to jointly learn the scene representation along with a 3D dense feature field and a 2D feature extractor. We learn the underlying geometry of the scene with an implicit field through volumetric rendering and design our feature field to leverage intermediate geometric information encoded in the implicit field. Visual localization is then achieved by aligning the image-based features and the rendered volumetric features.
  arXiv  Detail & Related papers  (2024-06-12T17:51:53Z)
• CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition [73.51329037954866]
  We propose a robust global representation method with cross-image correlation awareness for visual place recognition. Our method uses the attention mechanism to correlate multiple images within a batch. Our method outperforms state-of-the-art methods by a large margin with significantly less training time.
  arXiv  Detail & Related papers  (2024-02-29T15:05:11Z)
• Weatherproofing Retrieval for Localization with Generative AI and Geometric Consistency [32.46493952272438]
  State-of-the-art visual localization approaches rely on a first image retrieval step. We improve this retrieval step and tailor it to the final localization task. We experimentally show that those changes translate into large improvements for the most challenging visual localization datasets.
  arXiv  Detail & Related papers  (2024-02-14T15:24:20Z)
• Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination [63.992213016011235]
  We propose a method for scene relighting under novel views by learning a neural precomputed radiance transfer function. Our method can be solely supervised on a set of real images of the scene under a single unknown lighting condition. Results show that the recovered disentanglement of scene parameters improves significantly over the current state of the art.
  arXiv  Detail & Related papers  (2022-07-27T16:07:48Z)
• Neural Groundplans: Persistent Neural Scene Representations from a Single Image [90.04272671464238]
  We present a method to map 2D image observations of a scene to a persistent 3D scene representation. We propose conditional neural groundplans as persistent and memory-efficient scene representations.
  arXiv  Detail & Related papers  (2022-07-22T17:41:24Z)
• Continual Learning for Image-Based Camera Localization [14.47046413243358]
  We study the problem of visual localization in a continual learning setup. Our results show that similar to the classification domain, non-stationary data induces catastrophic forgetting in deep networks for visual localization. We propose a new sampling method based on coverage score (Buff-CS) that adapts the existing sampling strategies in the buffering process to the problem of visual localization.
  arXiv  Detail & Related papers  (2021-08-20T11:18:05Z)
• Data Augmentation for Object Detection via Differentiable Neural Rendering [71.00447761415388]
  It is challenging to train a robust object detector when annotated data is scarce. Existing approaches to tackle this problem include semi-supervised learning that interpolates labeled data from unlabeled data. We introduce an offline data augmentation method for object detection, which semantically interpolates the training data with novel views.
  arXiv  Detail & Related papers  (2021-03-04T06:31:06Z)
• Unsupervised Metric Relocalization Using Transform Consistency Loss [66.19479868638925]
  Training networks to perform metric relocalization traditionally requires accurate image correspondences. We propose a self-supervised solution, which exploits a key insight: localizing a query image within a map should yield the same absolute pose, regardless of the reference image used for registration. We evaluate our framework on synthetic and real-world data, showing our approach outperforms other supervised methods when a limited amount of ground-truth information is available.
  arXiv  Detail & Related papers  (2020-11-01T19:24:27Z)
• Geometrically Mappable Image Features [85.81073893916414]
  Vision-based localization of an agent in a map is an important problem in robotics and computer vision. We propose a method that learns image features targeted for image-retrieval-based localization.
  arXiv  Detail & Related papers  (2020-03-21T15:36:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.