Name of Thrones: Evaluating How LLMs Rank Student Names, Race, and Gender in Status Hierarchies

• URL: http://arxiv.org/abs/2504.10797v1
• Date: Tue, 15 Apr 2025 01:47:39 GMT
• Title: Name of Thrones: Evaluating How LLMs Rank Student Names, Race, and Gender in Status Hierarchies
• Authors: Annabella Sakunkoo, Jonathan Sakunkoo,
• Abstract summary: We conduct a large-scale analysis of name variations across 5 ethnicities to examine how AI exhibits name biases.<n>Contrary to the common assumption that AI tends to favor Whites, we show that East and, in some contexts, South Asian names receive higher rankings.<n>Gender moderates biases, with girls facing unfair disadvantages in certain racial groups.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Across cultures, names tell a lot about their bearers as they carry deep personal and cultural significance. Names also serve as powerful signals of gender, race, and status in the social hierarchy - a pecking order in which individual positions shape others' expectations on their perceived competence and worth. With the widespread adoption of LLMs and as names are often an input for LLMs, it is crucial to evaluate whether LLMs may sort people into status positions based on first and last names and, if so, whether it is in an unfair, biased fashion. While prior work has primarily investigated biases in first names, little attention has been paid to last names and even less to the combined effects of first and last names. In this study, we conduct a large-scale analysis of name variations across 5 ethnicities to examine how AI exhibits name biases. Our study investigates three key characteristics of inequality and finds that LLMs reflect and reinforce status hierarchies based on names that signal gender and ethnicity as they encode differential expectations of competence, leadership, and economic potential. Contrary to the common assumption that AI tends to favor Whites, we show that East and, in some contexts, South Asian names receive higher rankings. We also disaggregate Asians, a population projected to be the largest immigrant group in the U.S. by 2055. Our results challenge the monolithic Asian model minority assumption, illustrating a more complex and stratified model of bias. Gender moderates biases, with girls facing unfair disadvantages in certain racial groups. Additionally, spanning cultural categories by adopting Western first names improves AI-perceived status for East and Southeast Asian students, particularly for girls. Our findings underscore the importance of intersectional and more nuanced understandings of race, gender, and mixed identities in the evaluation of LLMs.

Related papers

• Popular LLMs Amplify Race and Gender Disparities in Human Mobility [2.601262068492271]
  This study investigates whether large language models (LLMs) exhibit biases in predicting human mobility based on race and gender. We find that LLMs frequently reflect and amplify existing societal biases.
  arXiv  Detail & Related papers  (2024-11-18T19:41:20Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• On the Influence of Gender and Race in Romantic Relationship Prediction from Large Language Models [21.178861746240507]
  We study the presence of heteronormative biases and prejudice against interracial romantic relationships in large language models. We show that models are less likely to predict romantic relationships for (a) same-gender character pairs than different-gender pairs; and (b) intra/inter-racial character pairs involving Asian names as compared to Black, Hispanic, or White names.
  arXiv  Detail & Related papers  (2024-10-05T01:41:55Z)
• GenderCARE: A Comprehensive Framework for Assessing and Reducing Gender Bias in Large Language Models [73.23743278545321]
  Large language models (LLMs) have exhibited remarkable capabilities in natural language generation, but have also been observed to magnify societal biases. GenderCARE is a comprehensive framework that encompasses innovative Criteria, bias Assessment, Reduction techniques, and Evaluation metrics.
  arXiv  Detail & Related papers  (2024-08-22T15:35:46Z)
• Beyond Binary Gender Labels: Revealing Gender Biases in LLMs through Gender-Neutral Name Predictions [5.896505047270243]
  We introduce an additional gender category, i.e., "neutral", to study and address potential gender biases in large language models. We investigate the impact of adding birth years to enhance the accuracy of gender prediction.
  arXiv  Detail & Related papers  (2024-07-07T05:59:09Z)
• Do Large Language Models Discriminate in Hiring Decisions on the Basis of Race, Ethnicity, and Gender? [15.362940175441048]
  We examine whether large language models (LLMs) exhibit race- and gender-based name discrimination in hiring decisions. We design a series of templatic prompts to LLMs to write an email to a named job applicant informing them of a hiring decision. By manipulating the applicant's first name, we measure the effect of perceived race, ethnicity, and gender on the probability that the LLM generates an acceptance or rejection email.
  arXiv  Detail & Related papers  (2024-06-15T03:31:16Z)
• The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring [0.9499648210774584]
  We conduct an AI audit of race and gender biases in one commonly-used large language model. We find that the model reflects some biases based on stereotypes. Women's resumes had occupations with less experience, while Asian and Hispanic resumes had immigrant markers.
  arXiv  Detail & Related papers  (2024-05-07T15:39:45Z)
• White Men Lead, Black Women Help? Benchmarking Language Agency Social Biases in LLMs [58.27353205269664]
  Social biases can manifest in language agency. We introduce the novel Language Agency Bias Evaluation benchmark. We unveil language agency social biases in 3 recent Large Language Model (LLM)-generated content.
  arXiv  Detail & Related papers  (2024-04-16T12:27:54Z)
• What's in a Name? Auditing Large Language Models for Race and Gender Bias [45.1187517058961]
  We employ an audit design to investigate biases in state-of-the-art large language models, including GPT-4.<n>We find that the advice systematically disadvantages names that are commonly associated with racial minorities and women.
  arXiv  Detail & Related papers  (2024-02-21T18:25:25Z)
• What Do Llamas Really Think? Revealing Preference Biases in Language Model Representations [62.91799637259657]
  Do large language models (LLMs) exhibit sociodemographic biases, even when they decline to respond? We study this research question by probing contextualized embeddings and exploring whether this bias is encoded in its latent representations. We propose a logistic Bradley-Terry probe which predicts word pair preferences of LLMs from the words' hidden vectors.
  arXiv  Detail & Related papers  (2023-11-30T18:53:13Z)
• Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs [67.51906565969227]
  We study the unintended side-effects of persona assignment on the ability of LLMs to perform basic reasoning tasks. Our study covers 24 reasoning datasets, 4 LLMs, and 19 diverse personas (e.g. an Asian person) spanning 5 socio-demographic groups.
  arXiv  Detail & Related papers  (2023-11-08T18:52:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.