CityGuessr: City-Level Video Geo-Localization on a Global Scale

• URL: http://arxiv.org/abs/2411.06344v1
• Date: Sun, 10 Nov 2024 03:20:00 GMT
• Title: CityGuessr: City-Level Video Geo-Localization on a Global Scale
• Authors: Parth Parag Kulkarni, Gaurav Kumar Nayak, Mubarak Shah,
• Abstract summary: We propose a novel problem of worldwide video geolocalization with the objective of hierarchically predicting the correct city, state/province, country, and continent, given a video. No large scale video datasets that have extensive worldwide coverage exist, to train models for solving this problem. We introduce a new dataset, CityGuessr68k comprising of 68,269 videos from 166 cities all over the world.
• Score: 54.371452373726584
• License:
• Abstract: Video geolocalization is a crucial problem in current times. Given just a video, ascertaining where it was captured from can have a plethora of advantages. The problem of worldwide geolocalization has been tackled before, but only using the image modality. Its video counterpart remains relatively unexplored. Meanwhile, video geolocalization has also garnered some attention in the recent past, but the existing methods are all restricted to specific regions. This motivates us to explore the problem of video geolocalization at a global scale. Hence, we propose a novel problem of worldwide video geolocalization with the objective of hierarchically predicting the correct city, state/province, country, and continent, given a video. However, no large scale video datasets that have extensive worldwide coverage exist, to train models for solving this problem. To this end, we introduce a new dataset, CityGuessr68k comprising of 68,269 videos from 166 cities all over the world. We also propose a novel baseline approach to this problem, by designing a transformer-based architecture comprising of an elegant Self-Cross Attention module for incorporating scenes as well as a TextLabel Alignment strategy for distilling knowledge from textlabels in feature space. To further enhance our location prediction, we also utilize soft-scene labels. Finally we demonstrate the performance of our method on our new dataset as well as Mapillary(MSLS). Our code and datasets are available at: https://github.com/ParthPK/CityGuessr

Related papers

• GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization [61.10806364001535]
  Worldwide Geo-localization aims to pinpoint the precise location of images taken anywhere on Earth. Existing approaches divide the globe into discrete geographic cells, transforming the problem into a classification task. We propose GeoCLIP, a novel CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between the image and its corresponding GPS locations.
  arXiv  Detail & Related papers  (2023-09-27T20:54:56Z)
• PIGEON: Predicting Image Geolocations [44.99833362998488]
  We present a new geolocalization system that combines semantic geocell creation, multi-task contrastive pretraining, and a novel loss function. PIGEOTTO is the first image geolocalization model that effectively generalizes to unseen places.
  arXiv  Detail & Related papers  (2023-07-11T23:36:49Z)
• Improving Toponym Resolution with Better Candidate Generation, Transformer-based Reranking, and Two-Stage Resolution [30.855736793066406]
  Geocoding is the task of converting location mentions in text into structured data that encodes the geospatial semantics. We propose a new architecture for geocoding, GeoNorm. Our proposed toponym resolution framework achieves state-of-the-art performance on multiple datasets.
  arXiv  Detail & Related papers  (2023-05-18T21:52:48Z)
• Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes [53.53712888703834]
  We introduce an end-to-end transformer-based architecture that exploits the relationship between different geographic levels. We achieve state of the art street level accuracy on 4 standard geo-localization datasets.
  arXiv  Detail & Related papers  (2023-03-07T21:47:58Z)
• G^3: Geolocation via Guidebook Grounding [92.46774241823562]
  We study explicit knowledge from human-written guidebooks that describe the salient and class-discriminative visual features humans use for geolocation. We propose the task of Geolocation via Guidebook Grounding that uses a dataset of StreetView images from a diverse set of locations. Our approach substantially outperforms a state-of-the-art image-only geolocation method, with an improvement of over 5% in Top-1 accuracy.
  arXiv  Detail & Related papers  (2022-11-28T16:34:40Z)
• GAMa: Cross-view Video Geo-localization [68.33955764543465]
  We focus on ground videos instead of images which provides contextual cues. At clip-level, a short video clip is matched with corresponding aerial image and is later used to get video-level geo-localization of a long video. Our proposed method achieves a Top-1 recall rate of 19.4% and 45.1% @1.0mile.
  arXiv  Detail & Related papers  (2022-07-06T04:25:51Z)
• Rethinking Visual Geo-localization for Large-Scale Applications [18.09618985653891]
  We build San Francisco eXtra Large, a new dataset covering a whole city and providing a wide range of challenging cases. We design a new highly scalable training technique, called CosPlace, which casts the training as a classification problem. We achieve state-of-the-art performance on a wide range of datasets and find that CosPlace is robust to heavy domain changes.
  arXiv  Detail & Related papers  (2022-04-05T15:33:45Z)
• TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization [81.70547404891099]
  CNN-based methods for cross-view image geo-localization fail to model global correlation. We propose a pure transformer-based approach (TransGeo) to address these limitations. TransGeo achieves state-of-the-art results on both urban and rural datasets.
  arXiv  Detail & Related papers  (2022-03-31T21:19:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.