PetalView: Fine-grained Location and Orientation Extraction of Street-view Images via Cross-view Local Search with Supplementary Materials

• URL: http://arxiv.org/abs/2406.13409v1
• Date: Wed, 19 Jun 2024 09:58:16 GMT
• Title: PetalView: Fine-grained Location and Orientation Extraction of Street-view Images via Cross-view Local Search with Supplementary Materials
• Authors: Wenmiao Hu, Yichen Zhang, Yuxuan Liang, Xianjing Han, Yifang Yin, Hannes Kruppa, See-Kiong Ng, Roger Zimmermann,
• Abstract summary: We present PetalView extractors with multi-scale search. PetalView extractors give semantically meaningful features that are equivalent across two drastically different views. When an angle prior is given, we propose a learnable prior angle mixer to utilize this information.
• Score: 40.49931158618459
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Satellite-based street-view information extraction by cross-view matching refers to a task that extracts the location and orientation information of a given street-view image query by using one or multiple geo-referenced satellite images. Recent work has initiated a new research direction to find accurate information within a local area covered by one satellite image centered at a location prior (e.g., from GPS). It can be used as a standalone solution or complementary step following a large-scale search with multiple satellite candidates. However, these existing works require an accurate initial orientation (angle) prior (e.g., from IMU) and/or do not efficiently search through all possible poses. To allow efficient search and to give accurate prediction regardless of the existence or the accuracy of the angle prior, we present PetalView extractors with multi-scale search. The PetalView extractors give semantically meaningful features that are equivalent across two drastically different views, and the multi-scale search strategy efficiently inspects the satellite image from coarse to fine granularity to provide sub-meter and sub-degree precision extraction. Moreover, when an angle prior is given, we propose a learnable prior angle mixer to utilize this information. Our method obtains the best performance on the VIGOR dataset and successfully improves the performance on KITTI dataset test 1 set with the recall within 1 meter (r@1m) for location estimation to 68.88% and recall within 1 degree (r@1d) 21.10% when no angle prior is available, and with angle prior achieves stable estimations at r@1m and r@1d above 70% and 21%, up to a 40-degree noise level.

Related papers

• FlightScope: A Deep Comprehensive Review of Aircraft Detection Algorithms in Satellite Imagery [2.9687381456164004]
  This paper critically evaluates and compares a suite of advanced object detection algorithms customized for the task of identifying aircraft within satellite imagery. This research encompasses an array of methodologies including YOLO versions 5 and 8, Faster RCNN, CenterNet, RetinaNet, RTMDet, and DETR, all trained from scratch. YOLOv5 emerges as a robust solution for aerial object detection, underlining its importance through superior mean average precision, Recall, and Intersection over Union scores.
  arXiv  Detail & Related papers  (2024-04-03T17:24:27Z)
• Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer [66.82008165644892]
  We propose a method to increase the accuracy of a ground camera's location and orientation by estimating the relative rotation and translation between the ground-level image and its matched/retrieved satellite image. Experimental results demonstrate that our method significantly outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2023-07-16T11:52:27Z)
• Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery with Supplementary Materials [29.015865103837413]
  An enormous amount of crowdsourced street-view images are uploaded to the internet. To prepare this hidden treasure for "ready-to-use" status, determining missing location information and camera orientation angles are two equally important tasks. Recent methods have achieved high performance on geo-localization of street-view images by cross-view matching with a pool of geo-referenced satellite imagery. In this work, we re-state the importance of finding fine-grained orientation for street-view images, formally define the problem and provide a set of evaluation metrics to assess the quality of the orientation estimation.
  arXiv  Detail & Related papers  (2023-07-07T05:50:14Z)
• Cross-View Visual Geo-Localization for Outdoor Augmented Reality [11.214903134756888]
  We address the problem of geo-pose estimation by cross-view matching of query ground images to a geo-referenced aerial satellite image database. We propose a new transformer neural network-based model and a modified triplet ranking loss for joint location and orientation estimation. Experiments on several benchmark cross-view geo-localization datasets show that our model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-03-28T01:58:03Z)
• An Adaptive Threshold for the Canny Edge Detection with Actor-Critic Algorithm [0.0]
  In deep learning-based object detection algorithms, the detection ability is superior to classical background subtraction (BGS) algorithms. This paper proposes a foreground-temporal fusion network (STFN) that could extract temporal and spatial information. The proposed algorithm shows 11.28% and 18.33% higher than the latest deep learning method in the LASIESTA and SBI dataset.
  arXiv  Detail & Related papers  (2022-09-19T01:15:32Z)
• CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization [89.69214577915959]
  This paper tackles the problem of Cross-view Video-based camera localization. We propose estimating the query camera's relative displacement to a satellite image before similarity matching. Experiments have demonstrated the effectiveness of video-based localization over single image-based localization.
  arXiv  Detail & Related papers  (2022-08-07T07:35:17Z)
• Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image [91.29546868637911]
  This paper addresses the problem of vehicle-mounted camera localization by matching a ground-level image with an overhead-view satellite map. The key idea is to formulate the task as pose estimation and solve it by neural-net based optimization. Experiments on standard autonomous vehicle localization datasets have confirmed the superiority of the proposed method.
  arXiv  Detail & Related papers  (2022-04-10T19:16:58Z)
• Accurate 3-DoF Camera Geo-Localization via Ground-to-Satellite Image Matching [102.39635336450262]
  We address the problem of ground-to-satellite image geo-localization by matching a query image captured at the ground level against a large-scale database with geotagged satellite images. Our new method is able to achieve the fine-grained location of a query image, up to pixel size precision of the satellite image.
  arXiv  Detail & Related papers  (2022-03-26T20:10:38Z)
• Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching [95.64702426906466]
  Cross-view geo-localization is a problem given a large-scale database of geo-tagged aerial images. Knowing orientation between ground and aerial images can significantly reduce matching ambiguity between these two views. We design a Dynamic Similarity Matching network to estimate cross-view orientation alignment during localization.
  arXiv  Detail & Related papers  (2020-05-08T05:21:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.