AI Sees Your Location, But With A Bias Toward The Wealthy World

• URL: http://arxiv.org/abs/2502.11163v3
• Date: Sat, 06 Sep 2025 18:47:00 GMT
• Title: AI Sees Your Location, But With A Bias Toward The Wealthy World
• Abstract summary: We introduce a benchmark consisting of 1,200 images paired with detailed geographic metadata.<n>We find that while these models demonstrate the ability to recognize geographic information from images, they exhibit significant biases.<n>Specifically, performance is substantially higher for economically developed and densely populated regions.
• Score: 21.42075535733345
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual-Language Models (VLMs) have shown remarkable performance across various tasks, particularly in recognizing geographic information from images. However, VLMs still show regional biases in this task. To systematically evaluate these issues, we introduce a benchmark consisting of 1,200 images paired with detailed geographic metadata. Evaluating four VLMs, we find that while these models demonstrate the ability to recognize geographic information from images, achieving up to 53.8% accuracy in city prediction, they exhibit significant biases. Specifically, performance is substantially higher for economically developed and densely populated regions compared to less developed (-12.5%) and sparsely populated (-17.0%) areas. Moreover, regional biases of frequently over-predicting certain locations remain. For instance, they consistently predict Sydney for images taken in Australia, shown by the low entropy scores for these countries. The strong performance of VLMs also raises privacy concerns, particularly for users who share images online without the intent of being identified. Our code and dataset are publicly available at https://github.com/uscnlp-lime/FairLocator.

Related papers

• Scaling Image Geo-Localization to Continent Level [48.7766435870634]
  This paper introduces a hybrid approach that achieves fine-grained geo-localization across a large geographic expanse the size of a continent.<n>We leverage a proxy classification task during training to learn rich feature representations that implicitly encode precise location information.<n>Our evaluation demonstrates that our approach can localize within 200m more than 68% of queries of a dataset covering a large part of Europe.
  arXiv  Detail & Related papers  (2025-10-30T17:59:35Z)
• Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales [61.03549470159347]
  Vision-language models (VLMs) have advanced rapidly, yet their capacity for image-grounded geolocation in open-world conditions has not been comprehensively evaluated.<n>We present EarthWhere, a comprehensive benchmark for VLM image geolocation that evaluates visual recognition, step-by-step reasoning, and evidence use.
  arXiv  Detail & Related papers  (2025-10-13T01:12:21Z)
• GeoArena: An Open Platform for Benchmarking Large Vision-language Models on WorldWide Image Geolocalization [21.941170274245223]
  Image geolocalization aims to predict the geographic location of images captured anywhere on Earth.<n>Current evaluation methodologies suffer from two major limitations.<n>We propose GeoArena, a first open platform for evaluating LVLMs on worldwide image geolocalization tasks.
  arXiv  Detail & Related papers  (2025-09-04T15:52:04Z)
• Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models [11.444835352261002]
  Geo-localization is the task of identifying the location of an image using visual cues alone.<n>Vision-Language Models (VLMs) are increasingly demonstrating capabilities as accurate image geo-locators.<n>This brings significant privacy risks, including those related to stalking and surveillance.
  arXiv  Detail & Related papers  (2025-08-27T15:21:31Z)
• From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models [14.178064117544082]
  Image geolocalization is important for applications in crisis response, digital forensics, and location-based intelligence.<n>Recent advances in large language models (LLMs) offer new opportunities for visual reasoning.<n>We introduce a benchmark called IMAGEO-Bench that systematically evaluates accuracy, distance error, geospatial bias, and reasoning process.
  arXiv  Detail & Related papers  (2025-08-03T06:04:33Z)
• Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views [3.611742324688716]
  We present KoreaGEO Bench, the first fine-grained, multimodal geolocation benchmark for Korean street views.<n>Our dataset comprises 1,080 high-resolution images sampled across four urban clusters and nine place types.<n>Results reveal modality-driven shifts in localization precision and highlight structural prediction biases toward core cities.
  arXiv  Detail & Related papers  (2025-06-03T20:28:55Z)
• Evaluating Precise Geolocation Inference Capabilities of Vision Language Models [0.0]
  This paper introduces a benchmark dataset collected from Google Street View that represents its global distribution of coverage. Foundation models are evaluated on single-image geolocation inference, with many achieving median distance errors of 300 km. We further evaluate VLM "agents" with access to supplemental tools, observing up to a 30.6% decrease in distance error.
  arXiv  Detail & Related papers  (2025-02-20T09:59:28Z)
• GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks [84.86699025256705]
  We present GEOBench-VLM, a benchmark specifically designed to evaluate Vision-Language Models (VLMs) on geospatial tasks.<n>Our benchmark features over 10,000 manually verified instructions and covers a diverse set of variations in visual conditions, object type, and scale.<n>We evaluate several state-of-the-art VLMs to assess their accuracy within the geospatial context.
  arXiv  Detail & Related papers  (2024-11-28T18:59:56Z)
• Image-Based Geolocation Using Large Vision-Language Models [19.071551941682063]
  We introduce tool, an innovative framework that significantly enhances image-based geolocation accuracy. tool employs a systematic chain-of-thought (CoT) approach, mimicking human geoguessing strategies. It achieves an impressive average score of 4550.5 in the GeoGuessr game, with an 85.37% win rate, and delivers highly precise geolocation predictions.
  arXiv  Detail & Related papers  (2024-08-18T13:39:43Z)
• Granular Privacy Control for Geolocation with Vision Language Models [36.3455665044992]
  We develop a new benchmark, GPTGeoChat, to test the ability of Vision Language Models to moderate geolocation dialogues with users. We collect a set of 1,000 image geolocation conversations between in-house annotators and GPT-4v. We evaluate the ability of various VLMs to moderate GPT-4v geolocation conversations by determining when too much location information has been revealed.
  arXiv  Detail & Related papers  (2024-07-06T04:06:55Z)
• Regional biases in image geolocation estimation: a case study with the SenseCity Africa dataset [0.0]
  We apply a state-of-the-art image geolocation estimation model (ISNs) to a crowd-sourced dataset of geolocated images from the African continent (SCA100) Our findings show that the ISNs model tends to over-predict image locations in high-income countries of the Western world. Our results suggest that using IM2GPS3k as a training set and benchmark for image geolocation estimation and other computer vision models overlooks its potential application in the African context.
  arXiv  Detail & Related papers  (2024-04-03T08:27:24Z)
• GeoLLM: Extracting Geospatial Knowledge from Large Language Models [49.20315582673223]
  We present GeoLLM, a novel method that can effectively extract geospatial knowledge from large language models. We demonstrate the utility of our approach across multiple tasks of central interest to the international community, including the measurement of population density and economic livelihoods. Our experiments reveal that LLMs are remarkably sample-efficient, rich in geospatial information, and robust across the globe.
  arXiv  Detail & Related papers  (2023-10-10T00:03:23Z)
• GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization [61.10806364001535]
  Worldwide Geo-localization aims to pinpoint the precise location of images taken anywhere on Earth. Existing approaches divide the globe into discrete geographic cells, transforming the problem into a classification task. We propose GeoCLIP, a novel CLIP-inspired Image-to-GPS retrieval approach that enforces alignment between the image and its corresponding GPS locations.
  arXiv  Detail & Related papers  (2023-09-27T20:54:56Z)
• PIGEON: Predicting Image Geolocations [44.99833362998488]
  We present a new geolocalization system that combines semantic geocell creation, multi-task contrastive pretraining, and a novel loss function. PIGEOTTO is the first image geolocalization model that effectively generalizes to unseen places.
  arXiv  Detail & Related papers  (2023-07-11T23:36:49Z)
• Inspecting the Geographical Representativeness of Images from Text-to-Image Models [52.80961012689933]
  We measure the geographical representativeness of generated images using a crowdsourced study comprising 540 participants across 27 countries. For deliberately underspecified inputs without country names, the generated images most reflect the surroundings of the United States followed by India. The overall scores for many countries still remain low, highlighting the need for future models to be more geographically inclusive.
  arXiv  Detail & Related papers  (2023-05-18T16:08:11Z)
• GeoNet: Benchmarking Unsupervised Adaptation across Geographies [71.23141626803287]
  We study the problem of geographic robustness and make three main contributions. First, we introduce a large-scale dataset GeoNet for geographic adaptation. Second, we hypothesize that the major source of domain shifts arise from significant variations in scene context. Third, we conduct an extensive evaluation of several state-of-the-art unsupervised domain adaptation algorithms and architectures.
  arXiv  Detail & Related papers  (2023-03-27T17:59:34Z)
• Geographic and Geopolitical Biases of Language Models [43.62238334380897]
  We propose an approach to study the geographic bias (and knowledge) present in pretrained language models (PLMs) Our findings suggest PLMs' representations map surprisingly well to the physical world in terms of country-to-country associations. Last, we explain how large PLMs despite exhibiting notions of geographical proximity, over-amplify geopoliticalitism at inference time.
  arXiv  Detail & Related papers  (2022-12-20T16:32:54Z)
• Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning [49.04866469947569]
  We construct a Geo-Diverse Visual Commonsense Reasoning dataset (GD-VCR) to test vision-and-language models' ability to understand cultural and geo-location-specific commonsense. We find that the performance of both models for non-Western regions including East Asia, South Asia, and Africa is significantly lower than that for Western region.
  arXiv  Detail & Related papers  (2021-09-14T17:52:55Z)
• PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation [87.50205728818601]
  We propose a PriorGuided Local (PGL) self-supervised model that learns the region-wise local consistency in the latent feature space. Our PGL model learns the distinctive representations of local regions, and hence is able to retain structural information.
  arXiv  Detail & Related papers  (2020-11-25T11:03:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.