Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts

• URL: http://arxiv.org/abs/2501.04316v2
• Date: Thu, 04 Sep 2025 15:53:10 GMT
• Title: Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts
• Authors: Preethi Seshadri, Hongyu Chen, Sameer Singh, Seraphina Goldfarb-Tarrant,
• Abstract summary: Large language models (LLMs) are increasingly being deployed in high-stakes applications like hiring.<n>This work examines the allocational fairness of LLM-based hiring systems through two tasks that reflect actual HR usage.
• Score: 19.20592062296075
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are increasingly being deployed in high-stakes applications like hiring, yet their potential for unfair decision-making remains understudied in generative and retrieval settings. In this work, we examine the allocational fairness of LLM-based hiring systems through two tasks that reflect actual HR usage: resume summarization and applicant ranking. By constructing a synthetic resume dataset with controlled perturbations and curating job postings, we investigate whether model behavior differs across demographic groups. Our findings reveal that generated summaries exhibit meaningful differences more frequently for race than for gender perturbations. Models also display non-uniform retrieval selection patterns across demographic groups and exhibit high ranking sensitivity to both gender and race perturbations. Surprisingly, retrieval models can show comparable sensitivity to both demographic and non-demographic changes, suggesting that fairness issues may stem from broader model brittleness. Overall, our results indicate that LLM-based hiring systems, especially in the retrieval stage, can exhibit notable biases that lead to discriminatory outcomes in real-world contexts.

Related papers

• A Comprehensive Study of Implicit and Explicit Biases in Large Language Models [1.0555164678638427]
  This study highlights the need to address biases in Large Language Models amid growing generative AI.<n>We studied bias-specific benchmarks such as StereoSet and CrowSPairs to evaluate the existence of various biases in multiple generative models such as BERT and GPT 3.5.<n>Results indicated fine-tuned models struggle with gender biases but excelled at identifying and avoiding racial biases.
  arXiv  Detail & Related papers  (2025-11-18T05:27:17Z)
• An External Fairness Evaluation of LinkedIn Talent Search [55.18656975953939]
  We conduct an independent, third-party audit for bias of LinkedIn's Talent Search ranking system.<n>We focus on potential ranking bias across two attributes: gender and race.<n>Our analysis reveals an under-representation of minority groups in early ranks.
  arXiv  Detail & Related papers  (2025-11-13T19:10:49Z)
• Breaking the Benchmark: Revealing LLM Bias via Minimal Contextual Augmentation [12.56588481992456]
  Large Language Models have been shown to demonstrate stereotypical biases in their representations and behavior.<n>We introduce a novel and general augmentation framework that involves three plug-and-play steps.<n>We find that Large Language Models are susceptible to perturbations to their inputs, showcasing a higher likelihood to behave stereotypically.
  arXiv  Detail & Related papers  (2025-10-27T23:05:12Z)
• Addressing Bias in LLMs: Strategies and Application to Fair AI-based Recruitment [49.81946749379338]
  This work seeks to analyze the capacity of Transformers-based systems to learn demographic biases present in the data.<n>We propose a privacy-enhancing framework to reduce gender information from the learning pipeline as a way to mitigate biased behaviors in the final tools.
  arXiv  Detail & Related papers  (2025-06-13T15:29:43Z)
• Robustly Improving LLM Fairness in Realistic Settings via Interpretability [0.16843915833103415]
  Anti-bias prompts fail when realistic contextual details are introduced.<n>We find that adding realistic context such as company names, culture descriptions from public careers pages, and selective hiring constraints induces significant racial and gender biases.<n>Our internal bias mitigation identifies race and gender-correlated directions and applies affine concept editing at inference time.
  arXiv  Detail & Related papers  (2025-06-12T17:34:38Z)
• Evaluating Bias in LLMs for Job-Resume Matching: Gender, Race, and Education [8.235367170516769]
  Large Language Models (LLMs) offer the potential to automate hiring by matching job descriptions with candidate resumes.<n>However, biases inherent in these models may lead to unfair hiring practices, reinforcing societal prejudices and undermining workplace diversity.<n>This study examines the performance and fairness of LLMs in job-resume matching tasks within the English language and U.S. context.
  arXiv  Detail & Related papers  (2025-03-24T22:11:22Z)
• Biased Heritage: How Datasets Shape Models in Facial Expression Recognition [13.77824359359967]
  We study bias propagation from datasets to trained models in image-based Facial Expression Recognition systems. We introduce new bias metrics specifically designed for multiclass problems with multiple demographic groups. Our findings suggest that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets.
  arXiv  Detail & Related papers  (2025-03-05T12:25:22Z)
• Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models [10.565316815513235]
  Large language models (LLMs) may still exhibit implicit biases when simulating human behavior.<n>We show that state-of-the-art LLMs exhibit significant sociodemographic disparities in nearly all simulations.<n>When comparing our findings to real-world disparities reported in empirical studies, we find that the biases we uncovered are directionally aligned but markedly amplified.
  arXiv  Detail & Related papers  (2025-01-29T05:21:31Z)
• How far can bias go? -- Tracing bias from pretraining data to alignment [54.51310112013655]
  This study examines the correlation between gender-occupation bias in pre-training data and their manifestation in LLMs.<n>Our findings reveal that biases present in pre-training data are amplified in model outputs.
  arXiv  Detail & Related papers  (2024-11-28T16:20:25Z)
• The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models [58.130894823145205]
  We center transgender, nonbinary, and other gender-diverse identities to investigate how alignment procedures interact with pre-existing gender-diverse bias. Our findings reveal that DPO-aligned models are particularly sensitive to supervised finetuning. We conclude with recommendations tailored to DPO and broader alignment practices.
  arXiv  Detail & Related papers  (2024-11-06T06:50:50Z)
• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Models [12.12628747941818]
  This paper presents a novel framework for benchmarking hierarchical gender hiring bias in Large Language Models (LLMs) for resume scoring. We introduce a new construct grounded in labour economics, legal principles, and critiques of current bias benchmarks. We analyze gender hiring biases in ten state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2024-06-17T09:15:57Z)
• Aligning with Whom? Large Language Models Have Gender and Racial Biases in Subjective NLP Tasks [15.015148115215315]
  We conduct experiments on four popular large language models (LLMs) to investigate their capability to understand group differences and potential biases in their predictions for politeness and offensiveness. We find that for both tasks, model predictions are closer to the labels from White and female participants. More specifically, when being prompted to respond from the perspective of "Black" and "Asian" individuals, models show lower performance in predicting both overall scores as well as the scores from corresponding groups.
  arXiv  Detail & Related papers  (2023-11-16T10:02:24Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization? [90.35044668396591]
  A recurring theme in machine learning is algorithmic monoculture: the same systems, or systems that share components, are deployed by multiple decision-makers. We propose the component-sharing hypothesis: if decision-makers share components like training data or specific models, then they will produce more homogeneous outcomes. We test this hypothesis on algorithmic fairness benchmarks, demonstrating that sharing training data reliably exacerbates homogenization. We conclude with philosophical analyses of and societal challenges for outcome homogenization, with an eye towards implications for deployed machine learning systems.
  arXiv  Detail & Related papers  (2022-11-25T09:33:11Z)
• Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairness [15.210232622716129]
  Data-driven predictive solutions predominant in commercial applications tend to suffer from biases and stereotypes. Data augmentation reduces gender bias by adding counterfactual examples to the training dataset. We show that some of the examples in the augmented dataset can be not important or even harmful for fairness.
  arXiv  Detail & Related papers  (2022-11-20T22:42:30Z)
• Data Representativeness in Accessibility Datasets: A Meta-Analysis [7.6597163467929805]
  We review datasets sourced by people with disabilities and older adults. We find that accessibility datasets represent diverse ages, but have gender and race representation gaps. We hope our effort expands the space of possibility for greater inclusion of marginalized communities in AI-infused systems.
  arXiv  Detail & Related papers  (2022-07-16T23:32:19Z)
• Balancing Biases and Preserving Privacy on Balanced Faces in the Wild [50.915684171879036]
  There are demographic biases present in current facial recognition (FR) models. We introduce our Balanced Faces in the Wild dataset to measure these biases across different ethnic and gender subgroups. We find that relying on a single score threshold to differentiate between genuine and imposters sample pairs leads to suboptimal results. We propose a novel domain adaptation learning scheme that uses facial features extracted from state-of-the-art neural networks.
  arXiv  Detail & Related papers  (2021-03-16T15:05:49Z)
• Enhancing Facial Data Diversity with Style-based Face Aging [59.984134070735934]
  In particular, face datasets are typically biased in terms of attributes such as gender, age, and race. We propose a novel, generative style-based architecture for data augmentation that captures fine-grained aging patterns. We show that the proposed method outperforms state-of-the-art algorithms for age transfer.
  arXiv  Detail & Related papers  (2020-06-06T21:53:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.