Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network

• URL: http://arxiv.org/abs/2408.05475v1
• Date: Sat, 10 Aug 2024 08:03:58 GMT
• Title: Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network
• Authors: Junyan Ye, Zhutao Lv, Weijia Li, Jinhua Yu, Haote Yang, Huaping Zhong, Conghui He,
• Abstract summary: Cross-view geolocalization identifies the geographic location of street view images by matching them with a georeferenced satellite database. We propose a new approach for cross-view image geo-localization, i.e., the Panorama-BEV Co-Retrieval Network.
• Score: 12.692812966686066
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cross-view geolocalization identifies the geographic location of street view images by matching them with a georeferenced satellite database. Significant challenges arise due to the drastic appearance and geometry differences between views. In this paper, we propose a new approach for cross-view image geo-localization, i.e., the Panorama-BEV Co-Retrieval Network. Specifically, by utilizing the ground plane assumption and geometric relations, we convert street view panorama images into the BEV view, reducing the gap between street panoramas and satellite imagery. In the existing retrieval of street view panorama images and satellite images, we introduce BEV and satellite image retrieval branches for collaborative retrieval. By retaining the original street view retrieval branch, we overcome the limited perception range issue of BEV representation. Our network enables comprehensive perception of both the global layout and local details around the street view capture locations. Additionally, we introduce CVGlobal, a global cross-view dataset that is closer to real-world scenarios. This dataset adopts a more realistic setup, with street view directions not aligned with satellite images. CVGlobal also includes cross-regional, cross-temporal, and street view to map retrieval tests, enabling a comprehensive evaluation of algorithm performance. Our method excels in multiple tests on common cross-view datasets such as CVUSA, CVACT, VIGOR, and our newly introduced CVGlobal, surpassing the current state-of-the-art approaches. The code and datasets can be found at \url{https://github.com/yejy53/EP-BEV}.

Related papers

• C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation [27.870926763424848]
  We propose a novel trainable retrieval architecture that uses bird's eye view (BEV) maps rather than vectors as embedding representation. Our method C-BEV surpasses the state-of-the-art on the retrieval task on multiple datasets by a large margin.
  arXiv  Detail & Related papers  (2023-12-13T11:14:57Z)
• GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization [61.10806364001535]
  Worldwide Geo-localization aims to pinpoint the precise location of images taken anywhere on Earth. Existing approaches divide the globe into discrete geographic cells, transforming the problem into a classification task. We propose GeoCLIP, a novel CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between the image and its corresponding GPS locations.
  arXiv  Detail & Related papers  (2023-09-27T20:54:56Z)
• Bird's-Eye-View Scene Graph for Vision-Language Navigation [85.72725920024578]
  Vision-language navigation (VLN) entails an agent to navigate 3D environments following human instructions. We present a BEV Scene Graph (BSG), which leverages multi-step BEV representations to encode scene layouts and geometric cues of indoor environment. Based on BSG, the agent predicts a local BEV grid-level decision score and a global graph-level decision score, combined with a sub-view selection score on panoramic views.
  arXiv  Detail & Related papers  (2023-08-09T07:48:20Z)
• Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes [53.53712888703834]
  We introduce an end-to-end transformer-based architecture that exploits the relationship between different geographic levels. We achieve state of the art street level accuracy on 4 standard geo-localization datasets.
  arXiv  Detail & Related papers  (2023-03-07T21:47:58Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization [89.69214577915959]
  This paper tackles the problem of Cross-view Video-based camera localization. We propose estimating the query camera's relative displacement to a satellite image before similarity matching. Experiments have demonstrated the effectiveness of video-based localization over single image-based localization.
  arXiv  Detail & Related papers  (2022-08-07T07:35:17Z)
• Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization [9.333087475006003]
  Cross-view image based geo-localization is notoriously challenging due to drastic viewpoint and appearance differences between the two domains. We show that we can address this discrepancy explicitly by learning to synthesize realistic street views from satellite inputs. We propose a novel multi-task architecture in which image synthesis and retrieval are considered jointly.
  arXiv  Detail & Related papers  (2021-03-11T17:40:59Z)
• Geometry-Guided Street-View Panorama Synthesis from Satellite Imagery [80.6282101835164]
  We present a new approach for synthesizing a novel street-view panorama given an overhead satellite image. Our method generates a Google's omnidirectional street-view type panorama, as if it is captured from the same geographical location as the center of the satellite patch.
  arXiv  Detail & Related papers  (2021-03-02T10:27:05Z)
• VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval [19.239311087570318]
  Cross-view image geo-localization aims to determine the locations of street-view query images by matching with GPS-tagged reference images from aerial view. Recent works have achieved surprisingly high retrieval accuracy on city-scale datasets. We propose a new large-scale benchmark -- VIGOR -- for cross-View Image Geo-localization beyond One-to-one Retrieval.
  arXiv  Detail & Related papers  (2020-11-24T15:50:54Z)
• AiRound and CV-BrCT: Novel Multi-View Datasets for Scene Classification [2.931113769364182]
  We present two new publicly available datasets named thedatasetand CV-BrCT. The first one contains triplets of images from the same geographic coordinate with different perspectives of view extracted from various places around the world. The second dataset contains pairs of aerial and street-level images extracted from southeast Brazil.
  arXiv  Detail & Related papers  (2020-08-03T18:55:46Z)
• Cross-View Image Retrieval -- Ground to Aerial Image Retrieval through Deep Learning [3.326320568999945]
  We present a novel cross-modal retrieval method specifically for multi-view images, called Cross-view Image Retrieval CVIR. Our approach aims to find a feature space as well as an embedding space in which samples from street-view images are compared directly to satellite-view images. For this comparison, a novel deep metric learning based solution "DeepCVIR" has been proposed.
  arXiv  Detail & Related papers  (2020-05-02T06:52:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.