Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval

• URL: http://arxiv.org/abs/2407.20371v2
• Date: Tue, 20 Aug 2024 21:49:26 GMT
• Title: Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval
• Authors: Kyra Wilson, Aylin Caliskan,
• Abstract summary: We investigate the possibilities of using large language models (LLMs) in a resume screening setting via a document retrieval framework. We then perform a resume audit study to determine whether a selection of Massive Text Embedding (MTE) models are biased in resume screening scenarios. We find that the MTEs are biased, significantly favoring White-associated names in 85.1% of cases and female-associated names in only 11.1% of cases.
• Score: 5.122502168590131
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Artificial intelligence (AI) hiring tools have revolutionized resume screening, and large language models (LLMs) have the potential to do the same. However, given the biases which are embedded within LLMs, it is unclear whether they can be used in this scenario without disadvantaging groups based on their protected attributes. In this work, we investigate the possibilities of using LLMs in a resume screening setting via a document retrieval framework that simulates job candidate selection. Using that framework, we then perform a resume audit study to determine whether a selection of Massive Text Embedding (MTE) models are biased in resume screening scenarios. We simulate this for nine occupations, using a collection of over 500 publicly available resumes and 500 job descriptions. We find that the MTEs are biased, significantly favoring White-associated names in 85.1\% of cases and female-associated names in only 11.1\% of cases, with a minority of cases showing no statistically significant differences. Further analyses show that Black males are disadvantaged in up to 100\% of cases, replicating real-world patterns of bias in employment settings, and validate three hypotheses of intersectionality. We also find an impact of document length as well as the corpus frequency of names in the selection of resumes. These findings have implications for widely used AI tools that are automating employment, fairness, and tech policy.

Related papers

• GenderBias-\emph{VL}: Benchmarking Gender Bias in Vision Language Models via Counterfactual Probing [72.0343083866144]
  This paper introduces the GenderBias-emphVL benchmark to evaluate occupation-related gender bias in Large Vision-Language Models. Using our benchmark, we extensively evaluate 15 commonly used open-source LVLMs and state-of-the-art commercial APIs. Our findings reveal widespread gender biases in existing LVLMs.
  arXiv  Detail & Related papers  (2024-06-30T05:55:15Z)
• JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Models [12.12628747941818]
  This paper presents a novel framework for benchmarking hierarchical gender hiring bias in Large Language Models (LLMs) for resume scoring. We introduce a new construct grounded in labour economics, legal principles, and critiques of current bias benchmarks. We analyze gender hiring biases in ten state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2024-06-17T09:15:57Z)
• The Silicon Ceiling: Auditing GPT's Race and Gender Biases in Hiring [0.9499648210774584]
  We conduct an algorithm audit of race and gender biases in one commonly-used large language model. We find that the model reflects some biases based on stereotypes. Women's resumes had occupations with less experience, while Asian and Hispanic resumes had immigrant markers.
  arXiv  Detail & Related papers  (2024-05-07T15:39:45Z)
• What's in a Name? Auditing Large Language Models for Race and Gender Bias [49.28899492966893]
  We employ an audit design to investigate biases in state-of-the-art large language models, including GPT-4. We find that the advice systematically disadvantages names that are commonly associated with racial minorities and women.
  arXiv  Detail & Related papers  (2024-02-21T18:25:25Z)
• Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes [73.12947922129261]
  We leverage the zero-shot capabilities of large language models to reduce stereotyping. We show that self-debiasing can significantly reduce the degree of stereotyping across nine different social groups. We hope this work opens inquiry into other zero-shot techniques for bias mitigation.
  arXiv  Detail & Related papers  (2024-02-03T01:40:11Z)
• Assessing Privacy Risks in Language Models: A Case Study on Summarization Tasks [65.21536453075275]
  We focus on the summarization task and investigate the membership inference (MI) attack. We exploit text similarity and the model's resistance to document modifications as potential MI signals. We discuss several safeguards for training summarization models to protect against MI attacks and discuss the inherent trade-off between privacy and utility.
  arXiv  Detail & Related papers  (2023-10-20T05:44:39Z)
• Are Emily and Greg Still More Employable than Lakisha and Jamal? Investigating Algorithmic Hiring Bias in the Era of ChatGPT [24.496590819263865]
  Large Language Models (LLMs) such as GPT-3.5, Bard, and Claude exhibit applicability across numerous tasks. We evaluate LLMs on two tasks: (1) matching resumes to job categories; and (2) summarizing resumes with employment relevant information. Overall, LLMs are robust across race and gender. They differ in their performance on pregnancy status and political affiliation.
  arXiv  Detail & Related papers  (2023-10-08T12:08:48Z)
• Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Models [0.0]
  This paper investigates bias along less-studied but still consequential, dimensions, such as age and beauty. We ask whether LLMs hold wide-reaching biases of positive or negative sentiment for specific social groups similar to the "what is beautiful is good" bias found in people in experimental psychology.
  arXiv  Detail & Related papers  (2023-09-16T07:07:04Z)
• Large Language Models Are Not Robust Multiple Choice Selectors [117.72712117510953]
  Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs) This work shows that modern LLMs are vulnerable to option position changes due to their inherent "selection bias" We propose a label-free, inference-time debiasing method, called PriDe, which separates the model's prior bias for option IDs from the overall prediction distribution.
  arXiv  Detail & Related papers  (2023-09-07T17:44:56Z)
• CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models [52.25049362267279]
  We present a Chinese Bias Benchmark dataset that consists of over 100K questions jointly constructed by human experts and generative language models. The testing instances in the dataset are automatically derived from 3K+ high-quality templates manually authored with stringent quality control. Extensive experiments demonstrate the effectiveness of the dataset in detecting model bias, with all 10 publicly available Chinese large language models exhibiting strong bias in certain categories.
  arXiv  Detail & Related papers  (2023-06-28T14:14:44Z)
• Learning Effective Representations for Person-Job Fit by Feature Fusion [4.884826427985207]
  Person-job fit is to match candidates and job posts on online recruitment platforms using machine learning algorithms. In this paper, we propose to learn comprehensive and effective representations of the candidates and job posts via feature fusion. Experiments over 10 months real data show that our solution outperforms existing methods with a large margin.
  arXiv  Detail & Related papers  (2020-06-12T09:02:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.