Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili

• URL: http://arxiv.org/abs/2608.03532v1
• Date: Tue, 04 Aug 2026 12:13:43 GMT
• Title: Cross-Lingual Bias in Large Language Models: A Comparative Analysis of English and Swahili
• Abstract summary: We investigate whether social biases generalise across languages by submitting 4,900 symmetric English--Swahili prompt pairs to GPT-5.2 and Gemini 2.5 Flash.<n>Our findings show that bias transforms rather than transfers.<n>These reinforce the idea that English-only bias audits do not produce adequate coverage for multilingual deployment.
• Score: 4.548136909830987
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models are increasingly deployed in multilingual contexts, yet safety alignment and bias evaluation remain overwhelmingly English-centric. We investigate whether social biases generalise across languages by submitting 4,900 symmetric English--Swahili prompt pairs to GPT-5.2 and Gemini 2.5 Flash across nine demographic bias axes, yielding 19,600 completions evaluated for stereotype prevalence, sentiment, refusal behaviour, and cross-lingual semantic similarity. Our findings show that bias transforms rather than transfers: stereotype rates shifted by up to 12 percentage points on specific axes, Gemini's neutral-sentiment rate doubled in Swahili, and GPT-5.2 refused 169 prompts in English and zero in Swahili, consistent with refusal behaviour anchored to English-language surface forms at the behavioural level. Over 55% of prompt pairs produced semantically dissimilar completions across both models. These reinforce the idea that English-only bias audits do not produce adequate coverage for multilingual deployment.

Related papers

• WinoQueer-NL: Assessing Bias in Dutch Language Models toward LGBTQ+ Identities [9.844587561630805]
  We developed a Dutch dataset based on the English WinoQueer benchmark, containing pairs of stereotypical and counter-stereotypical sentences.<n>We conducted an online survey with 43 Dutch queer participants, confirming 145 of 171 stereotypes as culturally relevant.<n>The final released dataset, comprising 42,906 sentences, was evaluated using a range of Dutch-specific and multilingual models.
  arXiv  Detail & Related papers  (2026-09-02T14:25:50Z)
• Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit [0.9699640804685629]
  We audit six large language models for gender stereotyping across English, Korean, Chinese, and Japanese.<n>Our findings show that their stereotyping spans a range roughly 2.5 times wider than the entire cross-country range found in humans.<n>One English-centric model, prompted in Korean, reached 5 times the local baseline, even when the prompt stated the candidate had already been hired.
  arXiv  Detail & Related papers  (2026-05-29T03:45:24Z)
• Bilingual Bias in Large Language Models: A Taiwan Sovereignty Benchmark Study [0.0]
  Large Language Models (LLMs) are increasingly deployed in multilingual contexts, yet their consistency across languages on politically sensitive topics remains understudied.<n>This paper presents a systematic benchmark study examining how 17 LLMs respond to questions concerning the sovereignty of the Republic of China (Taiwan) when queried in Chinese versus English.<n>We discover significant language bias -- the phenomenon where the same model produces substantively different political stances depending on the query language.
  arXiv  Detail & Related papers  (2026-02-06T03:57:21Z)
• When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training [57.230355403478995]
  We investigate the development of language-agnostic concept spaces during pretraining of EuroLLM.<n>We find that shared concept spaces emerge early and continue to refine, but that alignment with them is language-dependent.<n>In contrast to prior work, our fine-grained manual analysis reveals that some apparent gains in translation quality reflect shifts in behavior.
  arXiv  Detail & Related papers  (2026-01-30T11:23:01Z)
• Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation [49.2073409243885]
  Large language models (LLMs) excel at generating English counterfactuals and demonstrate multilingual proficiency.<n>We conduct automatic evaluations on both directly generated counterfactuals in the target languages and those derived via English translation across six languages.<n>We identify and categorize four main types of errors that consistently appear in the generated counterfactuals across languages.
  arXiv  Detail & Related papers  (2026-01-01T08:53:49Z)
• Beyond Early-Token Bias: Model-Specific and Language-Specific Position Effects in Multilingual LLMs [50.07451351559251]
  We present a study across five typologically distinct languages (English, Russian, German, Hindi, and Vietnamese)<n>We examine how position bias interacts with prompt strategies and affects output entropy.
  arXiv  Detail & Related papers  (2025-05-22T02:23:00Z)
• Crosslingual Reasoning through Test-Time Scaling [51.55526326294275]
  We find that scaling up inference compute for English-centric reasoning language models (RLMs) improves multilingual mathematical reasoning across many languages.<n>While English-centric RLM's CoTs are naturally predominantly English, they consistently follow a quote-and-think pattern to reason about quoted non-English inputs.<n>We observe poor out-of-domain reasoning generalization, in particular from STEM to cultural commonsense knowledge, even for English.
  arXiv  Detail & Related papers  (2025-05-08T16:50:06Z)
• Assessing Agentic Large Language Models in Multilingual National Bias [31.67058518564021]
  Cross-language disparities in reasoning-based recommendations remain largely unexplored.<n>This study is the first to address this gap.<n>We investigate multilingual bias in state-of-the-art LLMs by analyzing their responses to decision-making tasks across multiple languages.
  arXiv  Detail & Related papers  (2025-02-25T08:07:42Z)
• Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis [12.767209085664247]
  We study whether gender or racial biases are imported when using cross-lingual transfer. We find that systems using cross-lingual transfer usually become more biased than their monolingual counterparts. We also find racial biases to be much more prevalent than gender biases.
  arXiv  Detail & Related papers  (2023-05-22T04:37:49Z)
• Comparing Biases and the Impact of Multilingual Training across Multiple Languages [70.84047257764405]
  We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task. We adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender. Our results reveal similarities in bias expression such as favoritism of groups that are dominant in each language's culture.
  arXiv  Detail & Related papers  (2023-05-18T18:15:07Z)
• Language Contamination Explains the Cross-lingual Capabilities of English Pretrained Models [79.38278330678965]
  We find that common English pretraining corpora contain significant amounts of non-English text. This leads to hundreds of millions of foreign language tokens in large-scale datasets. We then demonstrate that even these small percentages of non-English data facilitate cross-lingual transfer for models trained on them.
  arXiv  Detail & Related papers  (2022-04-17T23:56:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.