Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation

• URL: http://arxiv.org/abs/2412.06781v1
• Date: Mon, 09 Dec 2024 18:59:04 GMT
• Title: Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation
• Authors: Nicolas Dufour, David Picard, Vicky Kalogeiton, Loic Landrieu,
• Abstract summary: Global visual geolocation predicts where an image was captured on Earth. In this paper, we aim to close the gap between traditional geolocalization and modern generative methods. Our model achieves state-of-the-art performance on three visual geolocation benchmarks.
• Score: 19.028122299569052
• License:
• Abstract: Global visual geolocation predicts where an image was captured on Earth. Since images vary in how precisely they can be localized, this task inherently involves a significant degree of ambiguity. However, existing approaches are deterministic and overlook this aspect. In this paper, we aim to close the gap between traditional geolocalization and modern generative methods. We propose the first generative geolocation approach based on diffusion and Riemannian flow matching, where the denoising process operates directly on the Earth's surface. Our model achieves state-of-the-art performance on three visual geolocation benchmarks: OpenStreetView-5M, YFCC-100M, and iNat21. In addition, we introduce the task of probabilistic visual geolocation, where the model predicts a probability distribution over all possible locations instead of a single point. We introduce new metrics and baselines for this task, demonstrating the advantages of our diffusion-based approach. Codes and models will be made available.

Related papers

• Swarm Intelligence in Geo-Localization: A Multi-Agent Large Vision-Language Model Collaborative Framework [51.26566634946208]
  We introduce smileGeo, a novel visual geo-localization framework. By inter-agent communication, smileGeo integrates the inherent knowledge of these agents with additional retrieved information. Results show that our approach significantly outperforms current state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-21T03:31:30Z)
• GOMAA-Geo: GOal Modality Agnostic Active Geo-localization [49.599465495973654]
  We consider the task of active geo-localization (AGL) in which an agent uses a sequence of visual cues observed during aerial navigation to find a target specified through multiple possible modalities. GOMAA-Geo is a goal modality active geo-localization agent for zero-shot generalization between different goal modalities.
  arXiv  Detail & Related papers  (2024-06-04T02:59:36Z)
• Mapping High-level Semantic Regions in Indoor Environments without Object Recognition [50.624970503498226]
  The present work proposes a method for semantic region mapping via embodied navigation in indoor environments. To enable region identification, the method uses a vision-to-language model to provide scene information for mapping. By projecting egocentric scene understanding into the global frame, the proposed method generates a semantic map as a distribution over possible region labels at each location.
  arXiv  Detail & Related papers  (2024-03-11T18:09:50Z)
• CurriculumLoc: Enhancing Cross-Domain Geolocalization through Multi-Stage Refinement [11.108860387261508]
  Visual geolocalization is a cost-effective and scalable task that involves matching one or more query images taken at some unknown location, to a set of geo-tagged reference images. We develop CurriculumLoc, a novel keypoint detection and description with global semantic awareness and a local geometric verification. We achieve new high recall@1 scores of 62.6% and 94.5% on ALTO, with two different distances metrics, respectively.
  arXiv  Detail & Related papers  (2023-11-20T08:40:01Z)
• GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization [61.10806364001535]
  Worldwide Geo-localization aims to pinpoint the precise location of images taken anywhere on Earth. Existing approaches divide the globe into discrete geographic cells, transforming the problem into a classification task. We propose GeoCLIP, a novel CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between the image and its corresponding GPS locations.
  arXiv  Detail & Related papers  (2023-09-27T20:54:56Z)
• Cross-View Visual Geo-Localization for Outdoor Augmented Reality [11.214903134756888]
  We address the problem of geo-pose estimation by cross-view matching of query ground images to a geo-referenced aerial satellite image database. We propose a new transformer neural network-based model and a modified triplet ranking loss for joint location and orientation estimation. Experiments on several benchmark cross-view geo-localization datasets show that our model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-03-28T01:58:03Z)
• G^3: Geolocation via Guidebook Grounding [92.46774241823562]
  We study explicit knowledge from human-written guidebooks that describe the salient and class-discriminative visual features humans use for geolocation. We propose the task of Geolocation via Guidebook Grounding that uses a dataset of StreetView images from a diverse set of locations. Our approach substantially outperforms a state-of-the-art image-only geolocation method, with an improvement of over 5% in Top-1 accuracy.
  arXiv  Detail & Related papers  (2022-11-28T16:34:40Z)
• Leveraging Selective Prediction for Reliable Image Geolocation [6.453278464902654]
  We define the task of image localizability, i.e. suitability of an image for geolocation. We propose a selective prediction methodology to address the task. By abstaining from predicting non-localizable images, we improve geolocation accuracy from 27.8% to 70.5% at the city-scale.
  arXiv  Detail & Related papers  (2021-11-23T15:46:12Z)
• Hierarchical Attention Fusion for Geo-Localization [7.544917072241684]
  We introduce a hierarchical attention fusion network using multi-scale features for geo-localization. We extract the hierarchical feature maps from a convolutional neural network (CNN) and organically fuse the extracted features for image representations. Our training is self-supervised using adaptive weights to control the attention of feature emphasis from each hierarchical level.
  arXiv  Detail & Related papers  (2021-02-18T07:07:03Z)
• Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization [54.00111565818903]
  Cross-view geo-localization is to spot images of the same geographic target from different platforms. Existing methods usually concentrate on mining the fine-grained feature of the geographic target in the image center. We introduce a simple and effective deep neural network, called Local Pattern Network (LPN), to take advantage of contextual information.
  arXiv  Detail & Related papers  (2020-08-26T16:06:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.