Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations

• URL: http://arxiv.org/abs/2502.13221v1
• Date: Tue, 18 Feb 2025 19:01:04 GMT
• Title: Two Tickets are Better than One: Fair and Accurate Hiring Under Strategic LLM Manipulations
• Authors: Lee Cohen, Jack Hsieh, Connie Hong, Judy Hanwen Shen,
• Abstract summary: We introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models.<n>We propose a two-ticket'' scheme, where the hiring algorithm applies an additional manipulation to each submitted resume.<n>We prove that hiring outcomes converge to a fixed, group-independent decision, eliminating disparities arising from differential access.
• Score: 4.536082020644864
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In an era of increasingly capable foundation models, job seekers are turning to generative AI tools to enhance their application materials. However, unequal access to and knowledge about generative AI tools can harm both employers and candidates by reducing the accuracy of hiring decisions and giving some candidates an unfair advantage. To address these challenges, we introduce a new variant of the strategic classification framework tailored to manipulations performed using large language models, accommodating varying levels of manipulations and stochastic outcomes. We propose a ``two-ticket'' scheme, where the hiring algorithm applies an additional manipulation to each submitted resume and considers this manipulated version together with the original submitted resume. We establish theoretical guarantees for this scheme, showing improvements for both the fairness and accuracy of hiring decisions when the true positive rate is maximized subject to a no false positives constraint. We further generalize this approach to an $n$-ticket scheme and prove that hiring outcomes converge to a fixed, group-independent decision, eliminating disparities arising from differential LLM access. Finally, we empirically validate our framework and the performance of our two-ticket scheme on real resumes using an open-source resume screening tool.

Related papers

• Evaluating the Promise and Pitfalls of LLMs in Hiring Decisions [1.1883838320818292]
  Large language models (LLMs) in hiring promise to streamline candidate screening, but it also raises serious concerns regarding accuracy and algorithmic bias.<n>We benchmark several state-of-the-art foundational LLMs and compare them with our proprietary domain-specific hiring model (Match Score) for job candidate matching.<n>Our experiments show that Match Score outperforms the general-purpose LLMs on accuracy (ROC AUC 0.85 vs 0.77) and achieves significantly more equitable outcomes across demographic groups.
  arXiv  Detail & Related papers  (2025-07-02T19:02:18Z)
• Self-Evolving Curriculum for LLM Reasoning [108.23021254812258]
  Self-Evolving Curriculum (SEC) is an automatic curriculum learning method that learns a curriculum policy concurrently with the RL fine-tuning process.<n>Our experiments demonstrate that SEC significantly improves models' reasoning capabilities, enabling better generalization to harder, out-of-distribution test problems.
  arXiv  Detail & Related papers  (2025-05-20T23:17:15Z)
• Why Ask One When You Can Ask $k$? Two-Stage Learning-to-Defer to the Top-$k$ Experts [3.6787328174619254]
  Learning-to-Defer (L2D) enables decision-making systems to improve reliability by selectively deferring uncertain predictions to more competent agents. We propose Top-$k$ Learning-to-Defer, a generalization of the classical two-stage L2D framework that allocates each query to the $k$ most confident agents instead of a single one. To further enhance flexibility and cost-efficiency, we introduce Top-$k(x)$ Learning-to-Defer, an adaptive extension that learns the optimal number of agents to consult for each query.
  arXiv  Detail & Related papers  (2025-04-17T14:50:40Z)
• Scalable Best-of-N Selection for Large Language Models via Self-Certainty [65.31658824274894]
  Best-of-N selection is a key technique for improving the reasoning performance of Large Language Models. We propose self-certainty, a novel and efficient metric to estimate response quality without requiring external reward models. Our findings establish self-certainty as a practical and efficient way for improving LLM reasoning capabilities.
  arXiv  Detail & Related papers  (2025-02-25T19:08:07Z)
• Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verification [52.095460362197336]
  Large language models (LLMs) struggle with consistent and accurate reasoning. LLMs are trained primarily on correct solutions, reducing their ability to detect and learn from errors. We propose a novel collaborative method integrating Chain-of-Thought (CoT) and Program-of-Thought (PoT) solutions for verification.
  arXiv  Detail & Related papers  (2024-10-05T05:21:48Z)
• Learning to Use Tools via Cooperative and Interactive Agents [58.77710337157665]
  Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. We propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. Our experiments on three datasets show that the LLMs, when equipped with ConAgents, outperform baselines with substantial improvement.
  arXiv  Detail & Related papers  (2024-03-05T15:08:16Z)
• Learning Fair Ranking Policies via Differentiable Optimization of Ordered Weighted Averages [55.04219793298687]
  This paper shows how efficiently-solvable fair ranking models can be integrated into the training loop of Learning to Rank. In particular, this paper is the first to show how to backpropagate through constrained optimizations of OWA objectives, enabling their use in integrated prediction and decision models.
  arXiv  Detail & Related papers  (2024-02-07T20:53:53Z)
• Weakly Supervised Learners for Correction of AI Errors with Provable Performance Guarantees [38.36817319051697]
  We present a new methodology for handling AI errors by introducing weakly supervised AI error correctors with a priori performance guarantees. These AI correctors are auxiliary maps whose role is to moderate the decisions of some previously constructed underlying classifier by either approving or rejecting its decisions. A key technical focus of the work is in providing performance guarantees for these new AI correctors through bounds on the probabilities of incorrect decisions.
  arXiv  Detail & Related papers  (2024-01-31T20:36:13Z)
• On Leveraging Large Language Models for Enhancing Entity Resolution: A Cost-efficient Approach [7.996010840316654]
  We propose an uncertainty reduction framework using Large Language Models (LLMs) to improve entity resolution results. LLMs capitalize on their advanced linguistic capabilities and a pay-as-you-go'' model that provides significant advantages to those without extensive data science expertise. We show that our method is efficient and effective, offering promising applications in real-world tasks.
  arXiv  Detail & Related papers  (2024-01-07T09:06:58Z)
• Optimizing Credit Limit Adjustments Under Adversarial Goals Using Reinforcement Learning [42.303733194571905]
  We seek to find and automatize an optimal credit card limit adjustment policy by employing reinforcement learning techniques. Our research establishes a conceptual structure for applying reinforcement learning framework to credit limit adjustment.
  arXiv  Detail & Related papers  (2023-06-27T16:10:36Z)
• Leaving the Nest: Going Beyond Local Loss Functions for Predict-Then-Optimize [57.22851616806617]
  We show that our method achieves state-of-the-art results in four domains from the literature. Our approach outperforms the best existing method by nearly 200% when the localness assumption is broken.
  arXiv  Detail & Related papers  (2023-05-26T11:17:45Z)
• Canary in a Coalmine: Better Membership Inference with Ensembled Adversarial Queries [53.222218035435006]
  We use adversarial tools to optimize for queries that are discriminative and diverse. Our improvements achieve significantly more accurate membership inference than existing methods.
  arXiv  Detail & Related papers  (2022-10-19T17:46:50Z)
• RAGUEL: Recourse-Aware Group Unfairness Elimination [2.720659230102122]
  'Algorithmic recourse' offers feasible recovery actions to change unwanted outcomes. We introduce the notion of ranked group-level recourse fairness. We develop a'recourse-aware ranking' solution that satisfies ranked recourse fairness constraints.
  arXiv  Detail & Related papers  (2022-08-30T11:53:38Z)
• Individually Fair Learning with One-Sided Feedback [15.713330010191092]
  We consider an online learning problem with one-sided feedback, in which the learner is able to observe the true label only for positively predicted instances. On each round, $k$ instances arrive and receive classification outcomes according to a randomized policy deployed by the learner. We then construct an efficient reduction from our problem of online learning with one-sided feedback and a panel reporting fairness violations to the contextual semi-bandit problem.
  arXiv  Detail & Related papers  (2022-06-09T12:59:03Z)
• MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learning [65.52675802289775]
  We show that an uncertainty aware classifier can solve challenging reinforcement learning problems. We propose a novel method for computing the normalized maximum likelihood (NML) distribution. We show that the resulting algorithm has a number of intriguing connections to both count-based exploration methods and prior algorithms for learning reward functions.
  arXiv  Detail & Related papers  (2021-07-15T08:19:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.