A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores

• URL: http://arxiv.org/abs/2002.08035v2
• Date: Thu, 20 Feb 2020 06:13:16 GMT
• Title: A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
• Authors: Maria De-Arteaga, Riccardo Fogliato, Alexandra Chouldechova
• Abstract summary: We study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We first show that humans do alter their behavior when the tool is deployed. We show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk.
• Score: 85.12096045419686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The increased use of algorithmic predictions in sensitive domains has been accompanied by both enthusiasm and concern. To understand the opportunities and risks of these technologies, it is key to study how experts alter their decisions when using such tools. In this paper, we study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We focus on the question: Are humans capable of identifying cases in which the machine is wrong, and of overriding those recommendations? We first show that humans do alter their behavior when the tool is deployed. Then, we show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk, even when overriding the recommendation requires supervisory approval. These results highlight the risks of full automation and the importance of designing decision pipelines that provide humans with autonomy.

Related papers

• Designing Algorithmic Recommendations to Achieve Human-AI Complementarity [2.4247752614854203]
  We formalize the design of recommendation algorithms that assist human decision-makers. We use a potential-outcomes framework to model the effect of recommendations on a human decision-maker's binary treatment choice. We derive minimax optimal recommendation algorithms that can be implemented with machine learning.
  arXiv  Detail & Related papers  (2024-05-02T17:15:30Z)
• Does AI help humans make better decisions? A statistical evaluation framework for experimental and observational studies [0.43981305860983716]
  We show how to compare the performance of three alternative decision-making systems--human-alone, human-with-AI, and AI-alone. We find that the risk assessment recommendations do not improve the classification accuracy of a judge's decision to impose cash bail.
  arXiv  Detail & Related papers  (2024-03-18T01:04:52Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions [80.34972679938483]
  We introduce Conformal Decision Theory, a framework for producing safe autonomous decisions despite imperfect machine learning predictions. Decisions produced by our algorithms are safe in the sense that they come with provable statistical guarantees of having low risk. Experiments demonstrate the utility of our approach in robot motion planning around humans, automated stock trading, and robot manufacturing.
  arXiv  Detail & Related papers  (2023-10-09T17:59:30Z)
• Auditing for Human Expertise [12.967730957018688]
  We develop a statistical framework under which we can pose this question as a natural hypothesis test. We propose a simple procedure which tests whether expert predictions are statistically independent from the outcomes of interest. A rejection of our test thus suggests that human experts may add value to any algorithm trained on the available data.
  arXiv  Detail & Related papers  (2023-06-02T16:15:24Z)
• Algorithmic Assistance with Recommendation-Dependent Preferences [2.864550757598007]
  We consider the effect and design of algorithmic recommendations when they affect choices. We show that recommendation-dependent preferences create inefficiencies where the decision-maker is overly responsive to the recommendation.
  arXiv  Detail & Related papers  (2022-08-16T09:24:47Z)
• Doubting AI Predictions: Influence-Driven Second Opinion Recommendation [92.30805227803688]
  We propose a way to augment human-AI collaboration by building on a common organizational practice: identifying experts who are likely to provide complementary opinions. The proposed approach aims to leverage productive disagreement by identifying whether some experts are likely to disagree with an algorithmic assessment.
  arXiv  Detail & Related papers  (2022-04-29T20:35:07Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• On the Fairness of Machine-Assisted Human Decisions [3.4069627091757178]
  We show that the inclusion of a biased human decision-maker can revert common relationships between the structure of the algorithm and the qualities of resulting decisions. In the lab experiment, we demonstrate how predictions informed by gender-specific information can reduce average gender disparities in decisions.
  arXiv  Detail & Related papers  (2021-10-28T17:24:45Z)
• Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making [53.62514158534574]
  We study whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. We show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making.
  arXiv  Detail & Related papers  (2020-01-07T15:33:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.