An External Fairness Evaluation of LinkedIn Talent Search

• URL: http://arxiv.org/abs/2511.10752v1
• Date: Thu, 13 Nov 2025 19:10:49 GMT
• Title: An External Fairness Evaluation of LinkedIn Talent Search
• Authors: Tina Behzad, Siddartha Devic, Vatsal Sharan, Aleksandra Korolova, David Kempe,
• Abstract summary: We conduct an independent, third-party audit for bias of LinkedIn's Talent Search ranking system.<n>We focus on potential ranking bias across two attributes: gender and race.<n>Our analysis reveals an under-representation of minority groups in early ranks.
• Score: 55.18656975953939
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We conduct an independent, third-party audit for bias of LinkedIn's Talent Search ranking system, focusing on potential ranking bias across two attributes: gender and race. To do so, we first construct a dataset of rankings produced by the system, collecting extensive Talent Search results across a diverse set of occupational queries. We then develop a robust labeling pipeline that infers the two demographic attributes of interest for the returned users. To evaluate potential biases in the collected dataset of real-world rankings, we utilize two exposure disparity metrics: deviation from group proportions and MinSkew. Our analysis reveals an under-representation of minority groups in early ranks across many queries. We further examine potential causes of this disparity, and discuss why they may be difficult or, in some cases, impossible to fully eliminate among the early ranks of queries. Beyond static metrics, we also investigate the concept of subgroup fairness over time, highlighting temporal disparities in exposure and retention, which are often more difficult to audit for in practice. In employer recruiting platforms such as LinkedIn Talent Search, the persistence of a particular candidate over multiple days in the ranking can directly impact the probability that the given candidate is selected for opportunities. Our analysis reveals demographic disparities in this temporal stability, with some groups experiencing greater volatility in their ranked positions than others. We contextualize all our findings alongside LinkedIn's published self-audits of its Talent Search system and reflect on the methodological constraints of a black-box external evaluation, including limited observability and noisy demographic inference.

Related papers

• FAIRE: Assessing Racial and Gender Bias in AI-Driven Resume Evaluations [3.9681649902019136]
  We introduce a benchmark, FAIRE, to test for racial and gender bias in large language models (LLMs) used to evaluate resumes.<n>Our findings reveal that while every model exhibits some degree of bias, the magnitude and direction vary considerably.<n>It highlights the urgent need for strategies to reduce bias in AI-driven recruitment.
  arXiv  Detail & Related papers  (2025-04-02T07:11:30Z)
• Small Changes, Large Consequences: Analyzing the Allocational Fairness of LLMs in Hiring Contexts [19.20592062296075]
  Large language models (LLMs) are increasingly being deployed in high-stakes applications like hiring.<n>This work examines the allocational fairness of LLM-based hiring systems through two tasks that reflect actual HR usage.
  arXiv  Detail & Related papers  (2025-01-08T07:28:10Z)
• Auditing for Bias in Ad Delivery Using Inferred Demographic Attributes [50.37313459134418]
  We study the effects of inference error on auditing for bias in one prominent application: black-box audit of ad delivery using paired ads.<n>We propose a way to mitigate the inference error when evaluating skew in ad delivery algorithms.
  arXiv  Detail & Related papers  (2024-10-30T18:57:03Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• D-BIAS: A Causality-Based Human-in-the-Loop System for Tackling Algorithmic Bias [57.87117733071416]
  We propose D-BIAS, a visual interactive tool that embodies human-in-the-loop AI approach for auditing and mitigating social biases. A user can detect the presence of bias against a group by identifying unfair causal relationships in the causal network. For each interaction, say weakening/deleting a biased causal edge, the system uses a novel method to simulate a new (debiased) dataset.
  arXiv  Detail & Related papers  (2022-08-10T03:41:48Z)
• Individually Fair Learning with One-Sided Feedback [15.713330010191092]
  We consider an online learning problem with one-sided feedback, in which the learner is able to observe the true label only for positively predicted instances. On each round, $k$ instances arrive and receive classification outcomes according to a randomized policy deployed by the learner. We then construct an efficient reduction from our problem of online learning with one-sided feedback and a panel reporting fairness violations to the contextual semi-bandit problem.
  arXiv  Detail & Related papers  (2022-06-09T12:59:03Z)
• Fair Group-Shared Representations with Normalizing Flows [68.29997072804537]
  We develop a fair representation learning algorithm which is able to map individuals belonging to different groups in a single group. We show experimentally that our methodology is competitive with other fair representation learning algorithms.
  arXiv  Detail & Related papers  (2022-01-17T10:49:49Z)
• An Outcome Test of Discrimination for Ranked Lists [0.18416014644193063]
  We show that non-discrimination implies a system of moment inequalities. We show how to statistically test the implied inequalities, and validate our approach in an application using data from LinkedIn.
  arXiv  Detail & Related papers  (2021-11-15T16:42:57Z)
• Incentives for Item Duplication under Fair Ranking Policies [69.14168955766847]
  We study the behaviour of different fair ranking policies in the presence of duplicates. We find that fairness-aware ranking policies may conflict with diversity, due to their potential to incentivize duplication more than policies solely focused on relevance.
  arXiv  Detail & Related papers  (2021-10-29T11:11:15Z)
• Intersectional Affirmative Action Policies for Top-k Candidates Selection [3.4961413413444817]
  We study the problem of selecting the top-k candidates from a pool of applicants, where each candidate is associated with a score indicating his/her aptitude. We consider a situation in which some groups of candidates experience historical and present disadvantage that makes their chances of being accepted much lower than other groups. We propose two algorithms to solve this problem, analyze them, and evaluate them experimentally using a dataset of university application scores and admissions to bachelor degrees in an OECD country.
  arXiv  Detail & Related papers  (2020-07-29T12:27:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.