FG$^2$: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching

• URL: http://arxiv.org/abs/2503.18725v1
• Date: Mon, 24 Mar 2025 14:34:20 GMT
• Title: FG$^2$: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching
• Authors: Zimin Xia, Alexandre Alahi,
• Abstract summary: We propose a novel fine-grained cross-view localization method that estimates the 3 Degrees of Freedom pose of a ground-level image in an aerial image of the surroundings.<n>The pose is estimated by aligning a point plane generated from the ground image with a point plane sampled from the aerial image.<n>Compared to the previous state-of-the-art, our method reduces the mean localization error by 28% on the VIGOR cross-area test set.
• Score: 69.81167130510333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a novel fine-grained cross-view localization method that estimates the 3 Degrees of Freedom pose of a ground-level image in an aerial image of the surroundings by matching fine-grained features between the two images. The pose is estimated by aligning a point plane generated from the ground image with a point plane sampled from the aerial image. To generate the ground points, we first map ground image features to a 3D point cloud. Our method then learns to select features along the height dimension to pool the 3D points to a Bird's-Eye-View (BEV) plane. This selection enables us to trace which feature in the ground image contributes to the BEV representation. Next, we sample a set of sparse matches from computed point correspondences between the two point planes and compute their relative pose using Procrustes alignment. Compared to the previous state-of-the-art, our method reduces the mean localization error by 28% on the VIGOR cross-area test set. Qualitative results show that our method learns semantically consistent matches across ground and aerial views through weakly supervised learning from the camera pose.

Related papers

• BevSplat: Resolving Height Ambiguity via Feature-Based Gaussian Primitives for Weakly-Supervised Cross-View Localization [11.50186721264038]
  This paper addresses the problem of weakly supervised cross-view localization.<n>The goal is to estimate the pose of a ground camera relative to a satellite image with noisy ground truth annotations.<n>We propose BevSplat, a novel method that resolves height ambiguity by using feature-based Gaussian primitives.
  arXiv  Detail & Related papers  (2025-02-13T08:54:04Z)
• C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation [27.870926763424848]
  We propose a novel trainable retrieval architecture that uses bird's eye view (BEV) maps rather than vectors as embedding representation. Our method C-BEV surpasses the state-of-the-art on the retrieval task on multiple datasets by a large margin.
  arXiv  Detail & Related papers  (2023-12-13T11:14:57Z)
• FocusTune: Tuning Visual Localization through Focus-Guided Sampling [61.79440120153917]
  FocusTune is a focus-guided sampling technique to improve the performance of visual localization algorithms. We demonstrate that FocusTune both improves or matches state-of-the-art performance whilst keeping ACE's appealing low storage and compute requirements. This combination of high performance and low compute and storage requirements is particularly promising for applications in areas like mobile robotics and augmented reality.
  arXiv  Detail & Related papers  (2023-11-06T04:58:47Z)
• Learning Dense Flow Field for Highly-accurate Cross-view Camera Localization [15.89357790711828]
  This paper addresses the problem of estimating the 3-DoF camera pose for a ground-level image with respect to a satellite image. We propose a novel end-to-end approach that leverages the learning of dense pixel-wise flow fields in pairs of ground and satellite images. Our approach reduces the median localization error by 89%, 19%, 80% and 35% on the KITTI, Ford multi-AV, VIGOR and Oxford RobotCar datasets.
  arXiv  Detail & Related papers  (2023-09-27T10:26:26Z)
• Ground Plane Matters: Picking Up Ground Plane Prior in Monocular 3D Object Detection [92.75961303269548]
  The ground plane prior is a very informative geometry clue in monocular 3D object detection (M3OD) We propose a Ground Plane Enhanced Network (GPENet) which resolves both issues at one go. Our GPENet can outperform other methods and achieve state-of-the-art performance, well demonstrating the effectiveness and the superiority of the proposed approach.
  arXiv  Detail & Related papers  (2022-11-03T02:21:35Z)
• SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling [75.957103837167]
  Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape. Existing works try to employ the global feature extracted from sketch to directly predict the 3D coordinates, but they usually suffer from losing fine details that are not faithful to the input sketch.
  arXiv  Detail & Related papers  (2022-08-14T16:37:51Z)
• Satellite Image Based Cross-view Localization for Autonomous Vehicle [59.72040418584396]
  This paper shows that by using an off-the-shelf high-definition satellite image as a ready-to-use map, we are able to achieve cross-view vehicle localization up to a satisfactory accuracy. Our method is validated on KITTI and Ford Multi-AV Seasonal datasets as ground view and Google Maps as the satellite view.
  arXiv  Detail & Related papers  (2022-07-27T13:16:39Z)
• TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learning [20.81202315793742]
  This paper develops a joint 2D-3D learning approach to reconstruct a local metric-semantic mesh at each camera maintained by a visual odometry algorithm. The mesh can be assembled into a global environment model to capture the terrain topology and semantics during online operation.
  arXiv  Detail & Related papers  (2022-04-23T05:18:39Z)
• Refer-it-in-RGBD: A Bottom-up Approach for 3D Visual Grounding in RGBD Images [69.5662419067878]
  Grounding referring expressions in RGBD image has been an emerging field. We present a novel task of 3D visual grounding in single-view RGBD image where the referred objects are often only partially scanned due to occlusion. Our approach first fuses the language and the visual features at the bottom level to generate a heatmap that localizes the relevant regions in the RGBD image. Then our approach conducts an adaptive feature learning based on the heatmap and performs the object-level matching with another visio-linguistic fusion to finally ground the referred object.
  arXiv  Detail & Related papers  (2021-03-14T11:18:50Z)
• Mesh Reconstruction from Aerial Images for Outdoor Terrain Mapping Using Joint 2D-3D Learning [12.741811850885309]
  This paper addresses outdoor terrain mapping using overhead images obtained from an unmanned aerial vehicle. Dense depth estimation from aerial images during flight is challenging. We develop a joint 2D-3D learning approach to reconstruct local meshes at each camera, which can be assembled into a global environment model.
  arXiv  Detail & Related papers  (2021-01-06T02:09:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.