Regional Bias in Large Language Models

• URL: http://arxiv.org/abs/2601.16349v1
• Date: Thu, 22 Jan 2026 22:22:23 GMT
• Title: Regional Bias in Large Language Models
• Authors: M P V S Gopinadh, Kappara Lakshmi Sindhu, Soma Sekhar Pandu Ranga Raju P, Yesaswini Swarna,
• Abstract summary: Regional bias in large language models (LLMs) is an emerging concern in AI fairness and global representation.<n>We evaluate ten prominent LLMs using prompts that probe forced-choice decisions between regions under contextually neutral scenarios.<n>We introduce FAZE, a prompt-based evaluation framework that measures regional bias on a 10-point scale, where higher scores indicate a stronger tendency to favor specific regions.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study investigates regional bias in large language models (LLMs), an emerging concern in AI fairness and global representation. We evaluate ten prominent LLMs: GPT-3.5, GPT-4o, Gemini 1.5 Flash, Gemini 1.0 Pro, Claude 3 Opus, Claude 3.5 Sonnet, Llama 3, Gemma 7B, Mistral 7B, and Vicuna-13B using a dataset of 100 carefully designed prompts that probe forced-choice decisions between regions under contextually neutral scenarios. We introduce FAZE, a prompt-based evaluation framework that measures regional bias on a 10-point scale, where higher scores indicate a stronger tendency to favor specific regions. Experimental results reveal substantial variation in bias levels across models, with GPT-3.5 exhibiting the highest bias score (9.5) and Claude 3.5 Sonnet scoring the lowest (2.5). These findings indicate that regional bias can meaningfully undermine the reliability, fairness, and inclusivity of LLM outputs in real-world, cross-cultural applications. This work contributes to AI fairness research by highlighting the importance of inclusive evaluation frameworks and systematic approaches for identifying and mitigating geographic biases in language models.

Related papers

• HALF: Harm-Aware LLM Fairness Evaluation Aligned with Deployment [52.374772443536045]
  HALF (Harm-Aware LLM Fairness) is a framework that assesses model bias in realistic applications and weighs the outcomes by harm severity.<n>We show that HALF exposes a clear gap between previous benchmarking success and deployment readiness.
  arXiv  Detail & Related papers  (2025-10-14T07:13:26Z)
• EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models [1.141545154221656]
  EvalMORAAL is a transparent chain-of-thought framework to evaluate moral alignment in 20 large language models.<n>We assess models on the World Values Survey (55 countries, 19 topics) and the PEW Global Attitudes Survey (39 countries, 8 topics)
  arXiv  Detail & Related papers  (2025-10-07T13:52:16Z)
• Evaluating Bias in Spoken Dialogue LLMs for Real-World Decisions and Recommendations [18.706521321659995]
  Paralinguistic features, such as age, gender, and accent, can affect model outputs.<n>Open-source models are more sensitive to age and gender, and recommendation tasks tend to amplify cross-group disparities.<n>This work provides the first systematic study of biases in end-to-end spoken dialogue models.
  arXiv  Detail & Related papers  (2025-09-27T16:21:22Z)
• An Empirical Analysis on Large Language Models in Debate Evaluation [10.677407097411768]
  We investigate the capabilities and inherent biases of advanced large language models (LLMs) such as GPT-3.5 and GPT-4 in the context of debate evaluation. We uncover a consistent bias in both GPT-3.5 and GPT-4 towards the second candidate response presented. We also uncover lexical biases in both GPT-3.5 and GPT-4, especially when label sets carry connotations such as numerical or sequential.
  arXiv  Detail & Related papers  (2024-05-28T18:34:53Z)
• Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluation [49.3814117521631]
  Standard benchmarks of bias and fairness in large language models (LLMs) measure the association between the user attributes stated or implied by a prompt.<n>We develop analogous RUTEd evaluations from three contexts of real-world use: children's bedtime stories, user personas, and English language learning exercises.<n>We find that standard bias metrics have no significant correlation with the more realistic bias metrics.
  arXiv  Detail & Related papers  (2024-02-20T01:49:15Z)
• Less is More: Fewer Interpretable Region via Submodular Subset Selection [54.07758302264416]
  This paper re-models the above image attribution problem as a submodular subset selection problem. We construct a novel submodular function to discover more accurate small interpretation regions. For correctly predicted samples, the proposed method improves the Deletion and Insertion scores with an average of 4.9% and 2.5% gain relative to HSIC-Attribution.
  arXiv  Detail & Related papers  (2024-02-14T13:30:02Z)
• Global-Liar: Factuality of LLMs over Time and Geographic Regions [3.715487408753612]
  This study evaluates the factual accuracy, stability, and biases in widely adopted GPT models, including GPT-3.5 and GPT-4. We introduce 'Global-Liar,' a dataset uniquely balanced in terms of geographic and temporal representation.
  arXiv  Detail & Related papers  (2024-01-31T13:57:24Z)
• Distortions in Judged Spatial Relations in Large Language Models [45.875801135769585]
  GPT-4 exhibited superior performance with 55 percent accuracy, followed by GPT-3.5 at 47 percent, and Llama-2 at 45 percent. The models identified the nearest cardinal direction in most cases, reflecting their associative learning mechanism.
  arXiv  Detail & Related papers  (2024-01-08T20:08:04Z)
• GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models [83.30078426829627]
  Large language models (LLMs) have gained popularity and are being widely adopted by a large user community. The existing evaluation methods have many constraints, and their results exhibit a limited degree of interpretability. We propose a bias evaluation framework named GPTBIAS that leverages the high performance of LLMs to assess bias in models.
  arXiv  Detail & Related papers  (2023-12-11T12:02:14Z)
• HERB: Measuring Hierarchical Regional Bias in Pre-trained Language Models [33.0987914452712]
  Regional bias in language models (LMs) is a long-standing global discrimination problem. This paper bridges the gap by analysing the regional bias learned by the pre-trained language models. We propose a HiErarchical Regional Bias evaluation method (HERB) to quantify the bias in pre-trained LMs.
  arXiv  Detail & Related papers  (2022-11-05T11:30:57Z)
• Prompting GPT-3 To Be Reliable [117.23966502293796]
  This work decomposes reliability into four facets: generalizability, fairness, calibration, and factuality. We find that GPT-3 outperforms smaller-scale supervised models by large margins on all these facets.
  arXiv  Detail & Related papers  (2022-10-17T14:52:39Z)
• LOGAN: Local Group Bias Detection by Clustering [86.38331353310114]
  We argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. We propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region.
  arXiv  Detail & Related papers  (2020-10-06T16:42:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.