Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations

• URL: http://arxiv.org/abs/2411.07320v1
• Date: Mon, 11 Nov 2024 19:25:25 GMT
• Title: Richer Output for Richer Countries: Uncovering Geographical Disparities in Generated Stories and Travel Recommendations
• Authors: Kirti Bhagat, Kinshuk Vasisht, Danish Pruthi,
• Abstract summary: We examine the impact of large language models for two common scenarios that require geographical knowledge: (a) travel recommendations and (b) geo-anchored story generation. Specifically, we study four popular language models, and across about $100$K travel requests, and $200$K story generations, we observe that travel recommendations corresponding to poorer countries are less unique with fewer location references.
• Score: 9.505918815853644
• License:
• Abstract: While a large body of work inspects language models for biases concerning gender, race, occupation and religion, biases of geographical nature are relatively less explored. Some recent studies benchmark the degree to which large language models encode geospatial knowledge. However, the impact of the encoded geographical knowledge (or lack thereof) on real-world applications has not been documented. In this work, we examine large language models for two common scenarios that require geographical knowledge: (a) travel recommendations and (b) geo-anchored story generation. Specifically, we study four popular language models, and across about $100$K travel requests, and $200$K story generations, we observe that travel recommendations corresponding to poorer countries are less unique with fewer location references, and stories from these regions more often convey emotions of hardship and sadness compared to those from wealthier nations.

Related papers

• Evaluation of Geographical Distortions in Language Models: A Crucial Step Towards Equitable Representations [2.825324306665133]
  This study focuses on biases related to geographical knowledge. We explore the connection between geography and language models by highlighting their tendency to misrepresent spatial information.
  arXiv  Detail & Related papers  (2024-04-26T13:22:28Z)
• On the Scaling Laws of Geographical Representation in Language Models [0.11510009152620666]
  We show that geographical knowledge is observable even for tiny models, and that it scales consistently as we increase the model size. Notably, we observe that larger language models cannot mitigate the geographical bias that is inherent to the training data.
  arXiv  Detail & Related papers  (2024-02-29T18:04:11Z)
• Large Language Models are Geographically Biased [47.88767211956144]
  We study what Large Language Models (LLMs) know about the world we live in through the lens of geography. We show various problematic geographic biases, which we define as systemic errors in geospatial predictions.
  arXiv  Detail & Related papers  (2024-02-05T02:32:09Z)
• Geo-Encoder: A Chunk-Argument Bi-Encoder Framework for Chinese Geographic Re-Ranking [61.60169764507917]
  Chinese geographic re-ranking task aims to find the most relevant addresses among retrieved candidates. We propose an innovative framework, namely Geo-Encoder, to more effectively integrate Chinese geographical semantics into re-ranking pipelines.
  arXiv  Detail & Related papers  (2023-09-04T13:44:50Z)
• GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark [56.08664336835741]
  We propose a GeoGraphic Language Understanding Evaluation benchmark, named GeoGLUE. We collect data from open-released geographic resources and introduce six natural language understanding tasks. We pro vide evaluation experiments and analysis of general baselines, indicating the effectiveness and significance of the GeoGLUE benchmark.
  arXiv  Detail & Related papers  (2023-05-11T03:21:56Z)
• GeoNet: Benchmarking Unsupervised Adaptation across Geographies [71.23141626803287]
  We study the problem of geographic robustness and make three main contributions. First, we introduce a large-scale dataset GeoNet for geographic adaptation. Second, we hypothesize that the major source of domain shifts arise from significant variations in scene context. Third, we conduct an extensive evaluation of several state-of-the-art unsupervised domain adaptation algorithms and architectures.
  arXiv  Detail & Related papers  (2023-03-27T17:59:34Z)
• Geographic and Geopolitical Biases of Language Models [43.62238334380897]
  We propose an approach to study the geographic bias (and knowledge) present in pretrained language models (PLMs) Our findings suggest PLMs' representations map surprisingly well to the physical world in terms of country-to-country associations. Last, we explain how large PLMs despite exhibiting notions of geographical proximity, over-amplify geopoliticalitism at inference time.
  arXiv  Detail & Related papers  (2022-12-20T16:32:54Z)
• Do Language Models Know the Way to Rome? [4.344337854565144]
  We exploit the fact that in geography, ground truths are available beyond local relations. We find that language models generally encode limited geographic information, but with larger models performing the best.
  arXiv  Detail & Related papers  (2021-09-16T13:28:16Z)
• Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning [49.04866469947569]
  We construct a Geo-Diverse Visual Commonsense Reasoning dataset (GD-VCR) to test vision-and-language models' ability to understand cultural and geo-location-specific commonsense. We find that the performance of both models for non-Western regions including East Asia, South Asia, and Africa is significantly lower than that for Western region.
  arXiv  Detail & Related papers  (2021-09-14T17:52:55Z)
• A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation [98.25464306634758]
  We propose to utilize commonsense knowledge from external knowledge bases to generate reasonable stories. We employ multi-task learning which combines a discriminative objective to distinguish true and fake stories. Our model can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.
  arXiv  Detail & Related papers  (2020-01-15T05:42:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.