Render-and-Compare: Cross-View 6 DoF Localization from Noisy Prior

• URL: http://arxiv.org/abs/2302.06287v2
• Date: Sat, 6 Jul 2024 04:37:14 GMT
• Title: Render-and-Compare: Cross-View 6 DoF Localization from Noisy Prior
• Authors: Shen Yan, Xiaoya Cheng, Yuxiang Liu, Juelin Zhu, Rouwan Wu, Yu Liu, Maojun Zhang,
• Abstract summary: In this work, we propose to go beyond the traditional ground-level setting and exploit the cross-view localization from aerial to ground. As no public dataset exists for the studied problem, we collect a new dataset that provides a variety of cross-view images from smartphones and drones. We develop a semi-automatic system to acquire ground-truth poses for query images.
• Score: 17.08552155321949
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the significant progress in 6-DoF visual localization, researchers are mostly driven by ground-level benchmarks. Compared with aerial oblique photography, ground-level map collection lacks scalability and complete coverage. In this work, we propose to go beyond the traditional ground-level setting and exploit the cross-view localization from aerial to ground. We solve this problem by formulating camera pose estimation as an iterative render-and-compare pipeline and enhancing the robustness through augmenting seeds from noisy initial priors. As no public dataset exists for the studied problem, we collect a new dataset that provides a variety of cross-view images from smartphones and drones and develop a semi-automatic system to acquire ground-truth poses for query images. We benchmark our method as well as several state-of-the-art baselines and demonstrate that our method outperforms other approaches by a large margin.

Related papers

• FaVoR: Features via Voxel Rendering for Camera Relocalization [23.7893950095252]
  Camera relocalization methods range from dense image alignment to direct camera pose regression from a query image. We propose a novel approach that leverages a globally sparse yet locally dense 3D representation of 2D features. By tracking and triangulating landmarks over a sequence of frames, we construct a sparse voxel map optimized to render image patch descriptors observed during tracking.
  arXiv  Detail & Related papers  (2024-09-11T18:58:16Z)
• Weakly-supervised deepfake localization in diffusion-generated images [4.548755617115687]
  We propose a weakly-supervised localization problem based on the Xception network as the backbone architecture. We show that the best performing detection method (based on local scores) is less sensitive to the looser supervision than to the mismatch in terms of dataset or generator.
  arXiv  Detail & Related papers  (2023-11-08T10:27:36Z)
• Learning Dense Flow Field for Highly-accurate Cross-view Camera Localization [15.89357790711828]
  This paper addresses the problem of estimating the 3-DoF camera pose for a ground-level image with respect to a satellite image. We propose a novel end-to-end approach that leverages the learning of dense pixel-wise flow fields in pairs of ground and satellite images. Our approach reduces the median localization error by 89%, 19%, 80% and 35% on the KITTI, Ford multi-AV, VIGOR and Oxford RobotCar datasets.
  arXiv  Detail & Related papers  (2023-09-27T10:26:26Z)
• Cross-View Image Sequence Geo-localization [6.555961698070275]
  Cross-view geo-localization aims to estimate the GPS location of a query ground-view image. Recent approaches use panoramic ground-view images to increase the range of visibility. We present the first cross-view geo-localization method that works on a sequence of limited Field-Of-View images.
  arXiv  Detail & Related papers  (2022-10-25T19:46:18Z)
• CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization [89.69214577915959]
  This paper tackles the problem of Cross-view Video-based camera localization. We propose estimating the query camera's relative displacement to a satellite image before similarity matching. Experiments have demonstrated the effectiveness of video-based localization over single image-based localization.
  arXiv  Detail & Related papers  (2022-08-07T07:35:17Z)
• Satellite Image Based Cross-view Localization for Autonomous Vehicle [59.72040418584396]
  This paper shows that by using an off-the-shelf high-definition satellite image as a ready-to-use map, we are able to achieve cross-view vehicle localization up to a satisfactory accuracy. Our method is validated on KITTI and Ford Multi-AV Seasonal datasets as ground view and Google Maps as the satellite view.
  arXiv  Detail & Related papers  (2022-07-27T13:16:39Z)
• 6D Camera Relocalization in Visually Ambiguous Extreme Environments [79.68352435957266]
  We propose a novel method to reliably estimate the pose of a camera given a sequence of images acquired in extreme environments such as deep seas or extraterrestrial terrains. Our method achieves comparable performance with state-of-the-art methods on the indoor benchmark (7-Scenes dataset) using only 20% training data.
  arXiv  Detail & Related papers  (2022-07-13T16:40:02Z)
• Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image [91.29546868637911]
  This paper addresses the problem of vehicle-mounted camera localization by matching a ground-level image with an overhead-view satellite map. The key idea is to formulate the task as pose estimation and solve it by neural-net based optimization. Experiments on standard autonomous vehicle localization datasets have confirmed the superiority of the proposed method.
  arXiv  Detail & Related papers  (2022-04-10T19:16:58Z)
• Wide-Depth-Range 6D Object Pose Estimation in Space [124.94794113264194]
  6D pose estimation in space poses unique challenges that are not commonly encountered in the terrestrial setting. One of the most striking differences is the lack of atmospheric scattering, allowing objects to be visible from a great distance. We propose a single-stage hierarchical end-to-end trainable network that is more robust to scale variations.
  arXiv  Detail & Related papers  (2021-04-01T08:39:26Z)
• Single View Metrology in the Wild [94.7005246862618]
  We present a novel approach to single view metrology that can recover the absolute scale of a scene represented by 3D heights of objects or camera height above the ground. Our method relies on data-driven priors learned by a deep network specifically designed to imbibe weakly supervised constraints from the interplay of the unknown camera with 3D entities such as object heights. We demonstrate state-of-the-art qualitative and quantitative results on several datasets as well as applications including virtual object insertion.
  arXiv  Detail & Related papers  (2020-07-18T22:31:33Z)
• Evaluation of Cross-View Matching to Improve Ground Vehicle Localization with Aerial Perception [17.349420462716886]
  Cross-view matching refers to the problem of finding the closest match for a given query ground view image to one from a database of aerial images. In this paper, we evaluate cross-view matching for the task of localizing a ground vehicle over a longer trajectory.
  arXiv  Detail & Related papers  (2020-03-13T23:59:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.