Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

• URL: http://arxiv.org/abs/2203.03334v1
• Date: Mon, 7 Mar 2022 12:25:44 GMT
• Title: Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors
• Authors: Florian Fervers, Sebastian Bullinger, Christoph Bodensteiner, Michael Arens, Rainer Stiefelhagen
• Abstract summary: We propose a novel method for geo-tracking in outdoor environments by registering a vehicle's sensor information with aerial imagery of an unseen target region. We train a model in a metric learning setting to extract visual features from ground and aerial images. Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos.
• Score: 25.87104194833264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper proposes a novel method for geo-tracking, i.e. continuous metric self-localization in outdoor environments by registering a vehicle's sensor information with aerial imagery of an unseen target region. Geo-tracking methods offer the potential to supplant noisy signals from global navigation satellite systems (GNSS) and expensive and hard to maintain prior maps that are typically used for this purpose. The proposed geo-tracking method aligns data from on-board cameras and lidar sensors with geo-registered orthophotos to continuously localize a vehicle. We train a model in a metric learning setting to extract visual features from ground and aerial images. The ground features are projected into a top-down perspective via the lidar points and are matched with the aerial features to determine the relative pose between vehicle and orthophoto. Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos. It exhibits strong generalization, is robust to changes in the environment and requires only geo-poses as ground truth. We evaluate our approach on the KITTI-360 dataset and achieve a mean absolute position error (APE) of 0.94m. We further compare with previous approaches on the KITTI odometry dataset and achieve state-of-the-art results on the geo-tracking task.

Related papers

• Game4Loc: A UAV Geo-Localization Benchmark from Game Data [0.0]
  We introduce a more practical UAV geo-localization task including partial matches of cross-view paired data. Experiments demonstrate the effectiveness of our data and training method for UAV geo-localization.
  arXiv  Detail & Related papers  (2024-09-25T13:33:28Z)
• View Consistent Purification for Accurate Cross-View Localization [59.48131378244399]
  This paper proposes a fine-grained self-localization method for outdoor robotics. The proposed method addresses limitations in existing cross-view localization methods. It is the first sparse visual-only method that enhances perception in dynamic environments.
  arXiv  Detail & Related papers  (2023-08-16T02:51:52Z)
• Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer [66.82008165644892]
  We propose a method to increase the accuracy of a ground camera's location and orientation by estimating the relative rotation and translation between the ground-level image and its matched/retrieved satellite image. Experimental results demonstrate that our method significantly outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2023-07-16T11:52:27Z)
• Energy-Based Models for Cross-Modal Localization using Convolutional Transformers [52.27061799824835]
  We present a novel framework for localizing a ground vehicle mounted with a range sensor against satellite imagery in the absence of GPS. We propose a method using convolutional transformers that performs accurate metric-level localization in a cross-modal manner. We train our model end-to-end and demonstrate our approach achieving higher accuracy than the state-of-the-art on KITTI, Pandaset, and a custom dataset.
  arXiv  Detail & Related papers  (2023-06-06T21:27:08Z)
• Cross-View Visual Geo-Localization for Outdoor Augmented Reality [11.214903134756888]
  We address the problem of geo-pose estimation by cross-view matching of query ground images to a geo-referenced aerial satellite image database. We propose a new transformer neural network-based model and a modified triplet ranking loss for joint location and orientation estimation. Experiments on several benchmark cross-view geo-localization datasets show that our model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-03-28T01:58:03Z)
• Uncertainty-aware Vision-based Metric Cross-view Geolocalization [25.87104194833264]
  We present an end-to-end differentiable model that uses the ground and aerial images to predict a probability distribution over possible vehicle poses. We improve the previous state-of-the-art by a large margin even without ground or aerial data from the test region.
  arXiv  Detail & Related papers  (2022-11-22T10:23:20Z)
• A Gis Aided Approach for Geolocalizing an Unmanned Aerial System Using Deep Learning [0.4297070083645048]
  We propose an alternative approach to geolocalize a UAS when GPS signal is degraded or denied. Considering UAS has a downward-looking camera on its platform that can acquire real-time images as the platform flies, we apply modern deep learning techniques to achieve geolocalization. We extract GIS information from OpenStreetMap (OSM) to semantically segment matched features into building and terrain classes.
  arXiv  Detail & Related papers  (2022-08-25T17:51:15Z)
• Satellite Image Based Cross-view Localization for Autonomous Vehicle [59.72040418584396]
  This paper shows that by using an off-the-shelf high-definition satellite image as a ready-to-use map, we are able to achieve cross-view vehicle localization up to a satisfactory accuracy. Our method is validated on KITTI and Ford Multi-AV Seasonal datasets as ground view and Google Maps as the satellite view.
  arXiv  Detail & Related papers  (2022-07-27T13:16:39Z)
• Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image [91.29546868637911]
  This paper addresses the problem of vehicle-mounted camera localization by matching a ground-level image with an overhead-view satellite map. The key idea is to formulate the task as pose estimation and solve it by neural-net based optimization. Experiments on standard autonomous vehicle localization datasets have confirmed the superiority of the proposed method.
  arXiv  Detail & Related papers  (2022-04-10T19:16:58Z)
• Accurate 3-DoF Camera Geo-Localization via Ground-to-Satellite Image Matching [102.39635336450262]
  We address the problem of ground-to-satellite image geo-localization by matching a query image captured at the ground level against a large-scale database with geotagged satellite images. Our new method is able to achieve the fine-grained location of a query image, up to pixel size precision of the satellite image.
  arXiv  Detail & Related papers  (2022-03-26T20:10:38Z)
• Real-time Geo-localization Using Satellite Imagery and Topography for Unmanned Aerial Vehicles [18.71806336611299]
  We propose a framework that is reliable in changing scenes and pragmatic for lightweight embedded systems on UAVs. The framework is comprised of two stages: offline database preparation and online inference. We present field experiments of image-based localization on two different UAV platforms to validate our results.
  arXiv  Detail & Related papers  (2021-08-07T01:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.