Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets

• URL: http://arxiv.org/abs/2602.09775v1
• Date: Tue, 10 Feb 2026 13:36:16 GMT
• Title: Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets
• Abstract summary: We geographically profile large-scale multimodal datasets by mapping image-caption pairs to countries based on location information extracted from captions using LLMs.<n>We find that the United States, the United Kingdom, and Canada account for $48.0%$ of samples, while South American and African countries are severely under-represented with only $1.8%$ and $3.8%$ of images, respectively.
• Score: 33.86868726260716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent studies show that text-to-image models often fail to generate geographically representative images, raising concerns about the representativeness of their training data and motivating the question: which parts of the world do these training examples come from? We geographically profile large-scale multimodal datasets by mapping image-caption pairs to countries based on location information extracted from captions using LLMs. Studying English captions from three widely used datasets (Re-LAION, DataComp1B, and Conceptual Captions) across $20$ common entities (e.g., house, flag), we find that the United States, the United Kingdom, and Canada account for $48.0\%$ of samples, while South American and African countries are severely under-represented with only $1.8\%$ and $3.8\%$ of images, respectively. We observe a strong correlation between a country's GDP and its representation in the data ($ρ= 0.82$). Examining non-English subsets for $4$ languages from the Re-LAION dataset, we find that representation skews heavily toward countries where these languages are predominantly spoken. Additionally, we find that higher representation does not necessarily translate to greater visual or semantic diversity. Finally, analyzing country-specific images generated by Stable Diffusion v1.3 trained on Re-LAION, we show that while generations appear realistic, they are severely limited in their coverage compared to real-world images.

Related papers

• GeoDiv: Framework For Measuring Geographical Diversity In Text-To-Image Models [42.84755778627983]
  Text-to-image (T2I) models are rapidly gaining popularity, yet their outputs often lack geographical diversity, reinforce stereotypes, and misrepresent regions.<n>We introduce GeoDiv, a framework leveraging large language and vision-language models to assess geographical diversity along two complementary axes.<n>GeoDiv reveals a consistent lack of diversity and identifies fine-grained attributes where models default to biased portrayals.
  arXiv  Detail & Related papers  (2026-02-25T17:08:43Z)
• BLIP3-KALE: Knowledge Augmented Large-Scale Dense Captions [118.35194230865451]
  We introduce BLIP3-KALE, a dataset of 218 million image-text pairs. KALE augments synthetic dense image captions with web-scale alt-text to generate factually grounded image captions. We train vision-language models on KALE and demonstrate improvements on vision-language tasks.
  arXiv  Detail & Related papers  (2024-11-12T00:52:52Z)
• Multilingual Diversity Improves Vision-Language Representations [97.16233528393356]
  Pre-training on this dataset outperforms using English-only or English-dominated datasets on ImageNet.<n>On a geographically diverse task like GeoDE, we also observe improvements across all regions, with the biggest gain coming from Africa.
  arXiv  Detail & Related papers  (2024-05-27T08:08:51Z)
• Inspecting the Geographical Representativeness of Images from Text-to-Image Models [52.80961012689933]
  We measure the geographical representativeness of generated images using a crowdsourced study comprising 540 participants across 27 countries. For deliberately underspecified inputs without country names, the generated images most reflect the surroundings of the United States followed by India. The overall scores for many countries still remain low, highlighting the need for future models to be more geographically inclusive.
  arXiv  Detail & Related papers  (2023-05-18T16:08:11Z)
• Geographic and Geopolitical Biases of Language Models [43.62238334380897]
  We propose an approach to study the geographic bias (and knowledge) present in pretrained language models (PLMs) Our findings suggest PLMs' representations map surprisingly well to the physical world in terms of country-to-country associations. Last, we explain how large PLMs despite exhibiting notions of geographical proximity, over-amplify geopoliticalitism at inference time.
  arXiv  Detail & Related papers  (2022-12-20T16:32:54Z)
• BAN-Cap: A Multi-Purpose English-Bangla Image Descriptions Dataset [0.5893124686141781]
  Resource-constrained languages like Bangla remain out of focus, predominantly due to a lack of standard datasets. We present a new dataset BAN-Cap following the widely used Flickr8k dataset, where we collect Bangla captions of the images provided by qualified annotators. We investigate the effect of text augmentation and demonstrate that an adaptive attention-based model combined with text augmentation using Contextualized Word Replacement (CWR) outperforms all state-of-the-art models for Bangla image captioning.
  arXiv  Detail & Related papers  (2022-05-28T15:39:09Z)
• Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision [57.031588264841]
  We leverage a noisy dataset of over one billion image alt-text pairs, obtained without expensive filtering or post-processing steps. A simple dual-encoder architecture learns to align visual and language representations of the image and text pairs using a contrastive loss. We show that the scale of our corpus can make up for its noise and leads to state-of-the-art representations even with such a simple learning scheme.
  arXiv  Detail & Related papers  (2021-02-11T10:08:12Z)
• Learning to Represent Image and Text with Denotation Graph [32.417311523031195]
  We propose learning representations from a set of implied, visually grounded expressions between image and text. We show that state-of-the-art multimodal learning models can be further improved by leveraging automatically harvested structural relations.
  arXiv  Detail & Related papers  (2020-10-06T18:00:58Z)
• PhraseCut: Language-based Image Segmentation in the Wild [62.643450401286]
  We consider the problem of segmenting image regions given a natural language phrase. Our dataset is collected on top of the Visual Genome dataset. Our experiments show that the scale and diversity of concepts in our dataset poses significant challenges to the existing state-of-the-art.
  arXiv  Detail & Related papers  (2020-08-03T20:58:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.