Humble AI in the real-world: the case of algorithmic hiring

• URL: http://arxiv.org/abs/2505.20918v1
• Date: Tue, 27 May 2025 09:09:38 GMT
• Title: Humble AI in the real-world: the case of algorithmic hiring
• Authors: Rahul Nair, Inge Vejsbjerg, Elizabeth Daly, Christos Varytimidis, Bran Knowles,
• Abstract summary: Humble AI argues for cautiousness in AI development and deployments through scepticism.<n>We present a real-world case study for humble AI in the domain of algorithmic hiring.
• Score: 9.53469974854897
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humble AI (Knowles et al., 2023) argues for cautiousness in AI development and deployments through scepticism (accounting for limitations of statistical learning), curiosity (accounting for unexpected outcomes), and commitment (accounting for multifaceted values beyond performance). We present a real-world case study for humble AI in the domain of algorithmic hiring. Specifically, we evaluate virtual screening algorithms in a widely used hiring platform that matches candidates to job openings. There are several challenges in misrecognition and stereotyping in such contexts that are difficult to assess through standard fairness and trust frameworks; e.g., someone with a non-traditional background is less likely to rank highly. We demonstrate technical feasibility of how humble AI principles can be translated to practice through uncertainty quantification of ranks, entropy estimates, and a user experience that highlights algorithmic unknowns. We describe preliminary discussions with focus groups made up of recruiters. Future user studies seek to evaluate whether the higher cognitive load of a humble AI system fosters a climate of trust in its outcomes.

Related papers

• An Approach to Grounding AI Model Evaluations in Human-derived Criteria [0.0]
  We propose a novel approach to augment existing benchmarks with human-derived evaluation criteria.<n>Grounding our study in the Perception Test and OpenEQA benchmarks, we conducted in-depth interviews and large-scale surveys.<n>Our findings reveal that participants perceive AI as lacking in interpretive and empathetic skills yet hold high expectations for AI performance.
  arXiv  Detail & Related papers  (2025-09-04T21:40:32Z)
• Conformal Prediction and Trustworthy AI [1.6574413179773761]
  Conformal predictors were developed in the 1990's by Gammerman, Vovk, and their research team, to provide set predictions with guaranteed confidence level.<n>Over recent years, they have grown in popularity and have become a mainstream methodology for uncertainty quantification in the machine learning community.<n>This article reviews the potential for conformal prediction to contribute to trustworthy AI beyond its marginal validity property.
  arXiv  Detail & Related papers  (2025-08-09T08:28:59Z)
• Bridging Ethical Principles and Algorithmic Methods: An Alternative Approach for Assessing Trustworthiness in AI Systems [0.0]
  This paper introduces an assessment method that combines the ethical components of Trustworthy AI with the algorithmic processes of PageRank and TrustRank.<n>The goal is to establish an assessment framework that minimizes the subjectivity inherent in the self-assessment techniques prevalent in the field.
  arXiv  Detail & Related papers  (2025-06-28T06:27:30Z)
• Behind the Screens: Uncovering Bias in AI-Driven Video Interview Assessments Using Counterfactuals [0.0]
  We introduce a counterfactual-based framework to evaluate and quantify bias in AI-driven personality assessments.<n>Our approach employs generative adversarial networks (GANs) to generate counterfactual representations of job applicants.<n>This work provides a scalable tool for fairness auditing of commercial AI hiring platforms.
  arXiv  Detail & Related papers  (2025-05-17T18:46:14Z)
• General Scales Unlock AI Evaluation with Explanatory and Predictive Power [57.7995945974989]
  benchmarking has guided progress in AI, but it has offered limited explanatory and predictive power for general-purpose AI systems.<n>We introduce general scales for AI evaluation that can explain what common AI benchmarks really measure.<n>Our fully-automated methodology builds on 18 newly-crafted rubrics that place instance demands on general scales that do not saturate.
  arXiv  Detail & Related papers  (2025-03-09T01:13:56Z)
• Follow the money: a startup-based measure of AI exposure across occupations, industries and regions [0.0]
  Existing measures of AI occupational exposure focus on AI's theoretical potential to substitute or complement human labour on the basis of technical feasibility.<n>We introduce the AI Startup Exposure (AISE) index-a novel metric based on occupational descriptions from O*NET and AI applications developed by startups.<n>Our findings suggest that AI adoption will be gradual and shaped by social factors as much as by the technical feasibility of AI applications.
  arXiv  Detail & Related papers  (2024-12-06T10:25:05Z)
• Engineering Trustworthy AI: A Developer Guide for Empirical Risk Minimization [53.80919781981027]
  Key requirements for trustworthy AI can be translated into design choices for the components of empirical risk minimization. We hope to provide actionable guidance for building AI systems that meet emerging standards for trustworthiness of AI.
  arXiv  Detail & Related papers  (2024-10-25T07:53:32Z)
• Bending the Automation Bias Curve: A Study of Human and AI-based Decision Making in National Security Contexts [0.0]
  We theorize about the relationship between background knowledge about AI, trust in AI, and how these interact with other factors to influence the probability of automation bias. We test these in a preregistered task identification experiment across a representative sample of 9000 adults in 9 countries with varying levels of AI industries.
  arXiv  Detail & Related papers  (2023-06-28T18:57:36Z)
• Guideline for Trustworthy Artificial Intelligence -- AI Assessment Catalog [0.0]
  It is clear that AI and business models based on it can only reach their full potential if AI applications are developed according to high quality standards. The issue of the trustworthiness of AI applications is crucial and is the subject of numerous major publications. This AI assessment catalog addresses exactly this point and is intended for two target groups.
  arXiv  Detail & Related papers  (2023-06-20T08:07:18Z)
• Fairness in AI and Its Long-Term Implications on Society [68.8204255655161]
  We take a closer look at AI fairness and analyze how lack of AI fairness can lead to deepening of biases over time. We discuss how biased models can lead to more negative real-world outcomes for certain groups. If the issues persist, they could be reinforced by interactions with other risks and have severe implications on society in the form of social unrest.
  arXiv  Detail & Related papers  (2023-04-16T11:22:59Z)
• Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment [66.91538273487379]
  There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data.
  arXiv  Detail & Related papers  (2023-02-13T16:44:44Z)
• Metaethical Perspectives on 'Benchmarking' AI Ethics [81.65697003067841]
  Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. We argue that it makes more sense to talk about 'values' rather than 'ethics' when considering the possible actions of present and future AI systems.
  arXiv  Detail & Related papers  (2022-04-11T14:36:39Z)
• Bias in Multimodal AI: Testbed for Fair Automatic Recruitment [73.85525896663371]
  We study how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data. We train automatic recruitment algorithms using a set of multimodal synthetic profiles consciously scored with gender and racial biases. Our methodology and results show how to generate fairer AI-based tools in general, and in particular fairer automated recruitment systems.
  arXiv  Detail & Related papers  (2020-04-15T15:58:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.