Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs

• URL: http://arxiv.org/abs/2604.19292v1
• Date: Tue, 21 Apr 2026 09:57:41 GMT
• Title: Location Not Found: Exposing Implicit Local and Global Biases in Multilingual LLMs
• Abstract summary: This work aims to quantify models' inter- and intra-lingual biases, via their ability to answer locale-ambiguous questions.<n>We present LocQA, a test set containing 2,156 questions in 12 languages, referring to various locale-dependent facts such as laws, dates, and measurements.<n>We show a global bias towards answers relevant to the US-locale, even when models are asked in languages other than English.<n>In Intra-lingually, we show that when multiple locales are relevant for the same language, models act as demographic probability engines, prioritizing locales with larger populations.
• Score: 35.2977350485249
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multilingual large language models (LLMs) have minimized the fluency gap between languages. This advancement, however, exposes models to the risk of biased behavior, as knowledge and norms may propagate across languages. In this work, we aim to quantify models' inter- and intra-lingual biases, via their ability to answer locale-ambiguous questions. To this end, we present LocQA, a test set containing 2,156 questions in 12 languages, referring to various locale-dependent facts such as laws, dates, and measurements. The questions do not contain indications of the locales they relate to, other than the querying language itself. LLMs' responses to LocQA locale-ambiguous questions thus reveal models' implicit priors. We used LocQA to evaluate 32 models, and detected two types of structural biases. Inter-lingually, we show a global bias towards answers relevant to the US-locale, even when models are asked in languages other than English. Moreover, we discovered that this global bias is exacerbated in models that underwent instruction tuning, compared to their base counterparts. Intra-lingually, we show that when multiple locales are relevant for the same language, models act as demographic probability engines, prioritizing locales with larger populations. Taken together, insights from LocQA may help in shaping LLMs' desired local behavior, and in quantifying the impact of various training phases on different kinds of biases.

Related papers

• Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG [55.258582772528506]
  We investigate whether the mixture of different document languages impacts generation and citation in unintended ways.<n>Across eight languages and six open-weight models, we find that models preferentially cite English sources when queries are in English.<n>We find that models sometimes trade-off document relevance for language preference, indicating that citation choices are not always driven by informativeness alone.
  arXiv  Detail & Related papers  (2025-09-17T12:58:18Z)
• XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering [48.913480244527925]
  Large Language Models (LLMs) have shown significant progress in Open-domain question answering (ODQA)<n>Most evaluations focus on English and assume locale-invariant answers across languages.<n>We introduce XLQA, a novel benchmark explicitly designed for locale-sensitive multilingual ODQA.
  arXiv  Detail & Related papers  (2025-08-22T07:00:13Z)
• Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMs [50.07451351559251]
  We present a study across five typologically distinct languages (English, Russian, German, Hindi, and Vietnamese)<n>We examine how position bias interacts with prompt strategies and affects output entropy.
  arXiv  Detail & Related papers  (2025-05-22T02:23:00Z)
• Assessing Agentic Large Language Models in Multilingual National Bias [31.67058518564021]
  Cross-language disparities in reasoning-based recommendations remain largely unexplored.<n>This study is the first to address this gap.<n>We investigate multilingual bias in state-of-the-art LLMs by analyzing their responses to decision-making tasks across multiple languages.
  arXiv  Detail & Related papers  (2025-02-25T08:07:42Z)
• Understanding and Mitigating Language Confusion in LLMs [76.96033035093204]
  We evaluate 15 typologically diverse languages with existing and newly-created English and multilingual prompts.<n>We find that Llama Instruct and Mistral models exhibit high degrees of language confusion.<n>We find that language confusion can be partially mitigated via few-shot prompting, multilingual SFT and preference tuning.
  arXiv  Detail & Related papers  (2024-06-28T17:03:51Z)
• Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models [62.91524967852552]
  Large language models (LLMs) are typically multilingual due to pretraining on diverse multilingual corpora.<n>But can these models relate corresponding concepts across languages, i.e., be crosslingual?<n>This study evaluates state-of-the-art LLMs on inherently crosslingual tasks.
  arXiv  Detail & Related papers  (2024-06-23T15:15:17Z)
• MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs [6.781972039785424]
  Generative large language models (LLMs) have been shown to exhibit harmful biases and stereotypes. We present MBBQ, a dataset that measures stereotypes commonly held across Dutch, Spanish, and Turkish languages. Our results confirm that some non-English languages suffer from bias more than English, even when controlling for cultural shifts.
  arXiv  Detail & Related papers  (2024-06-11T13:23:14Z)
• Evaluating the Elementary Multilingual Capabilities of Large Language Models with MultiQ [16.637598165238934]
  Large language models (LLMs) need to serve everyone, including a global majority of non-English speakers. Recent research shows that, despite limits in their intended use, people prompt LLMs in many different languages. We introduce MultiQ, a new silver standard benchmark for basic open-ended question answering with 27.4k test questions.
  arXiv  Detail & Related papers  (2024-03-06T16:01:44Z)
• Quantifying the Dialect Gap and its Correlates Across Languages [69.18461982439031]
  This work will lay the foundation for furthering the field of dialectal NLP by laying out evident disparities and identifying possible pathways for addressing them through mindful data collection.
  arXiv  Detail & Related papers  (2023-10-23T17:42:01Z)
• Locale Encoding For Scalable Multilingual Keyword Spotting Models [8.385848547707953]
  We propose two locale-conditioned universalmodels with locale feature concatenation and feature-wise linearmodulation. FiLM performed the best, improving on average FRRby 61% (relative) compared to monolingual KWS models of similarsizes.
  arXiv  Detail & Related papers  (2023-02-25T02:20:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.