Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit

• URL: http://arxiv.org/abs/2605.30804v1
• Date: Fri, 29 May 2026 03:45:24 GMT
• Title: Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit
• Abstract summary: We audit six large language models for gender stereotyping across English, Korean, Chinese, and Japanese.<n>Our findings show that their stereotyping spans a range roughly 2.5 times wider than the entire cross-country range found in humans.<n>One English-centric model, prompted in Korean, reached 5 times the local baseline, even when the prompt stated the candidate had already been hired.
• Score: 0.9699640804685629
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We audit six large language models (LLMs) for gender stereotyping across English, Korean, Chinese, and Japanese. Three were developed primarily for English-language use (Claude, GPT, Gemini) and three for East Asian use (DeepSeek, Syn-Pro, HyperCLOVA X). We adopt the HEXACO-100 personality inventory and anchor each model against a cross-cultural human dataset spanning 48 countries to ask not whether LLMs are biased, but how far their gender attributions drift from the populations they are deployed among. Our findings show that their stereotyping spans a range roughly 2.5 times wider than the entire cross-country range found in humans, and the effect can compound across languages. One English-centric model, prompted in Korean, reached 5 times the local baseline, even when the prompt stated the candidate had already been hired, which often dampens human stereotyping. To characterize such behaviors without ranking them, we introduce a four-pattern framework -- concordance, suppression, reorganization, and amplification -- across 24 (model x language) cells. Item-level analysis reveals that translation does not just rescale stereotypes, but changes the attributes tied to it, hiding significant rearrangement under the surface while appearing well-calibrated. Our results ultimately suggest that no single debiasing pipeline is likely to address bias evenly across linguistic boundaries.

Related papers

• Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models [81.45743826739054]
  A major barrier has been the lack of demographic annotations in web-scale datasets such as LAION-400M.<n>We create person-centric annotations for the full dataset, including over 276 million bounding boxes, perceived gender and race/ethnicity labels, and automatically generated captions.<n>Using them, we uncover demographic imbalances and harmful associations, such as the disproportionate linking of men and individuals perceived as Black or Middle Eastern with crime-related and negative content.
  arXiv  Detail & Related papers  (2025-10-04T07:51:59Z)
• Probing Gender Bias in Multilingual LLMs: A Case Study of Stereotypes in Persian [0.0]
  We propose a template-based probing methodology to uncover gender stereotypes in Multilingual Large Language Models (LLMs)<n>We evaluate four prominent models, focusing on Persian, a low-resource language with distinct linguistic features.<n>Our results show that all models exhibit gender stereotypes, with greater disparities in Persian than in English across all domains.<n>This study underscores the need for inclusive NLP practices and provides a framework for assessing bias in other low-resource languages.
  arXiv  Detail & Related papers  (2025-09-24T14:34:17Z)
• EuroGEST: Investigating gender stereotypes in multilingual language models [58.871032460235575]
  We introduce EuroGEST, a dataset designed to measure gender-stereotypical reasoning in LLMs across English and 29 European languages.<n>We show that the strongest stereotypes in all models across all languages are that women are 'beautiful', 'empathetic' and 'neat' and men are 'leaders','strong, tough' and 'professional'
  arXiv  Detail & Related papers  (2025-06-04T11:58:18Z)
• Measuring South Asian Biases in Large Language Models [1.5903891569492878]
  This work addresses gaps by conducting a multilingual and intersectional analysis of Large Language Models (LLMs)<n>We construct a culturally grounded bias lexicon capturing previously unexplored intersectional dimensions including gender, religion, marital status, and number of children.<n>We evaluate two self-debiasing strategies to measure their effectiveness in reducing culturally specific bias in Indo-Aryan and Dravidian languages.
  arXiv  Detail & Related papers  (2025-05-24T02:18:17Z)
• Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models [28.944990804599893]
  We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2.<n>We quantify race and gender bias and measure stereotype amplification.
  arXiv  Detail & Related papers  (2025-05-20T10:14:00Z)
• Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMs [15.783346695504344]
  We study the first study of multilingual intersecting country and gender biases.<n>We construct a benchmark of prompts in English, Spanish and German, using 25 countries and four pronoun sets.<n>We find that even when models show parity for gender or country individually, intersectional occupational biases based on both country and gender persist.
  arXiv  Detail & Related papers  (2025-05-05T08:40:51Z)
• Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You [64.74707085021858]
  We show that multilingual models suffer from significant gender biases just as monolingual models do.<n>We propose a novel benchmark, MAGBIG, intended to foster research on gender bias in multilingual models.<n>Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
  arXiv  Detail & Related papers  (2024-01-29T12:02:28Z)
• Multilingual large language models leak human stereotypes across language boundaries [25.903732543380528]
  We study how training a model multilingually may lead to stereotypes expressed in one language showing up in the models' behaviour in another. We propose a measurement framework for stereotype leakage and investigate its effect across English, Russian, Chinese, and Hindi. We find that GPT-3.5 exhibits the most stereotype leakage, and Hindi is the most susceptible to leakage effects.
  arXiv  Detail & Related papers  (2023-12-12T10:24:17Z)
• Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis [12.767209085664247]
  We study whether gender or racial biases are imported when using cross-lingual transfer. We find that systems using cross-lingual transfer usually become more biased than their monolingual counterparts. We also find racial biases to be much more prevalent than gender biases.
  arXiv  Detail & Related papers  (2023-05-22T04:37:49Z)
• Comparing Biases and the Impact of Multilingual Training across Multiple Languages [70.84047257764405]
  We present a bias analysis across Italian, Chinese, English, Hebrew, and Spanish on the downstream sentiment analysis task. We adapt existing sentiment bias templates in English to Italian, Chinese, Hebrew, and Spanish for four attributes: race, religion, nationality, and gender. Our results reveal similarities in bias expression such as favoritism of groups that are dominant in each language's culture.
  arXiv  Detail & Related papers  (2023-05-18T18:15:07Z)
• An Analysis of Social Biases Present in BERT Variants Across Multiple Languages [0.0]
  We investigate the bias present in monolingual BERT models across a diverse set of languages. We propose a template-based method to measure any kind of bias, based on sentence pseudo-likelihood. We conclude that current methods of probing for bias are highly language-dependent.
  arXiv  Detail & Related papers  (2022-11-25T23:38:08Z)
• Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Models [104.41668491794974]
  We quantify the usage of adjectives and verbs generated by language models surrounding the names of politicians as a function of their gender. We find that while some words such as dead, and designated are associated with both male and female politicians, a few specific words such as beautiful and divorced are predominantly associated with female politicians.
  arXiv  Detail & Related papers  (2021-04-15T15:03:26Z)
• How True is GPT-2? An Empirical Analysis of Intersectional Occupational Biases [50.591267188664666]
  Downstream applications are at risk of inheriting biases contained in natural language models. We analyze the occupational biases of a popular generative language model, GPT-2. For a given job, GPT-2 reflects the societal skew of gender and ethnicity in the US, and in some cases, pulls the distribution towards gender parity.
  arXiv  Detail & Related papers  (2021-02-08T11:10:27Z)
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer [101.58431011820755]
  We study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications. We create a multilingual dataset for bias analysis and propose several ways for quantifying bias in multilingual representations.
  arXiv  Detail & Related papers  (2020-05-02T04:34:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.