Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI

• URL: http://arxiv.org/abs/2508.07872v1
• Date: Mon, 11 Aug 2025 11:43:34 GMT
• Title: Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI
• Authors: Holli Sargeant, Mackenzie Jorgensen, Arina Shah, Adrian Weller, Umang Bhatt,
• Abstract summary: Uncertainty in artificial intelligence predictions poses urgent legal and ethical challenges for AI-assisted decision-making.<n>We examine two algorithmic interventions that act as guardrails for human-AI collaboration: selective abstention and selective friction.<n>We argue that although both interventions pose risks of unlawful discrimination under UK law, selective frictions offer a promising pathway toward fairer and more accountable AI-assisted decision-making.
• Score: 38.122893275090206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uncertainty in artificial intelligence (AI) predictions poses urgent legal and ethical challenges for AI-assisted decision-making. We examine two algorithmic interventions that act as guardrails for human-AI collaboration: selective abstention, which withholds high-uncertainty predictions from human decision-makers, and selective friction, which delivers those predictions together with salient warnings or disclosures that slow the decision process. Research has shown that selective abstention based on uncertainty can inadvertently exacerbate disparities and disadvantage under-represented groups that disproportionately receive uncertain predictions. In this paper, we provide the first integrated socio-technical and legal analysis of uncertainty-based algorithmic interventions. Through two case studies, AI-assisted consumer credit decisions and AI-assisted content moderation, we demonstrate how the seemingly neutral use of uncertainty thresholds can trigger discriminatory impacts. We argue that, although both interventions pose risks of unlawful discrimination under UK law, selective frictions offer a promising pathway toward fairer and more accountable AI-assisted decision-making by preserving transparency and encouraging more cautious human judgment.

Related papers

• Human aversion? Do AI Agents Judge Identity More Harshly Than Performance [0.06554326244334868]
  We investigate how AI agents based on large language models assess and integrate human input.<n>We find that the AI system systematically discounts human advice, penalizing human errors more severely than algorithmic errors.
  arXiv  Detail & Related papers  (2025-03-31T02:05:27Z)
• Human Decision-making is Susceptible to AI-driven Manipulation [87.24007555151452]
  AI systems may exploit users' cognitive biases and emotional vulnerabilities to steer them toward harmful outcomes.<n>This study examined human susceptibility to such manipulation in financial and emotional decision-making contexts.
  arXiv  Detail & Related papers  (2025-02-11T15:56:22Z)
• Human-Alignment Influences the Utility of AI-assisted Decision Making [16.732483972136418]
  We investigate what extent the degree of alignment actually influences the utility of AI-assisted decision making.<n>Our results show a positive association between the degree of alignment and the utility of AI-assisted decision making.
  arXiv  Detail & Related papers  (2025-01-23T19:01:47Z)
• Speculations on Uncertainty and Humane Algorithms [0.0]
  Provenance enables algorithms to know what they know preventing possible harms. It is essential to compute with what we know rather than make assumptions that may be unjustified or untenable.
  arXiv  Detail & Related papers  (2024-08-13T08:54:34Z)
• A Decision Theoretic Framework for Measuring AI Reliance [23.353778024330165]
  Humans frequently make decisions with the aid of artificially intelligent (AI) systems. Researchers have identified ensuring that a human has appropriate reliance on an AI as a critical component of achieving complementary performance. We propose a formal definition of reliance, based on statistical decision theory, which separates the concepts of reliance as the probability the decision-maker follows the AI's recommendation.
  arXiv  Detail & Related papers  (2024-01-27T09:13:09Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions [6.356355538824237]
  We argue that reliance and decision quality are often inappropriately conflated in the current literature on AI-assisted decision-making.<n>Our research highlights the importance of distinguishing between reliance behavior and decision quality in AI-assisted decision-making.
  arXiv  Detail & Related papers  (2023-04-18T08:08:05Z)
• Decision-makers Processing of AI Algorithmic Advice: Automation Bias versus Selective Adherence [0.0]
  Key concern is that human overreliance on algorithms introduces new biases in the human-algorithm interaction. A second concern regards decision-makers inclination to selectively adopt algorithmic advice when it matches their pre-existing beliefs and stereotypes. We assess these via two studies simulating the use of algorithmic advice in decisions pertaining to the employment of school teachers in the Netherlands. Our findings of selective, biased adherence belie the promise of neutrality that has propelled algorithm use in the public sector.
  arXiv  Detail & Related papers  (2021-03-03T13:10:50Z)
• Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty [66.17147341354577]
  We argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. We describe how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness.
  arXiv  Detail & Related papers  (2020-11-15T17:26:14Z)
• When Does Uncertainty Matter?: Understanding the Impact of Predictive Uncertainty in ML Assisted Decision Making [68.19284302320146]
  We carry out user studies to assess how people with differing levels of expertise respond to different types of predictive uncertainty. We found that showing posterior predictive distributions led to smaller disagreements with the ML model's predictions. This suggests that posterior predictive distributions can potentially serve as useful decision aids which should be used with caution and take into account the type of distribution and the expertise of the human.
  arXiv  Detail & Related papers  (2020-11-12T02:23:53Z)
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores [85.12096045419686]
  We study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We first show that humans do alter their behavior when the tool is deployed. We show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk.
  arXiv  Detail & Related papers  (2020-02-19T07:27:32Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.