Algorithmic risk assessments can alter human decision-making processes in high-stakes government contexts

• URL: http://arxiv.org/abs/2012.05370v1
• Date: Wed, 9 Dec 2020 23:44:45 GMT
• Title: Algorithmic risk assessments can alter human decision-making processes in high-stakes government contexts
• Authors: Ben Green, Yiling Chen
• Abstract summary: We show that risk assessments can alter decision-making processes by increasing the salience of risk as a factor in decisions and that these shifts could exacerbate racial disparities. These results demonstrate that improving human prediction accuracy with algorithms does not necessarily improve human decisions and highlight the need to experimentally test how government algorithms are used by human decision-makers.
• Score: 19.265010348250897
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Governments are increasingly turning to algorithmic risk assessments when making important decisions, believing that these algorithms will improve public servants' ability to make policy-relevant predictions and thereby lead to more informed decisions. Yet because many policy decisions require balancing risk-minimization with competing social goals, evaluating the impacts of risk assessments requires considering how public servants are influenced by risk assessments when making policy decisions rather than just how accurately these algorithms make predictions. Through an online experiment with 2,140 lay participants simulating two high-stakes government contexts, we provide the first large-scale evidence that risk assessments can systematically alter decision-making processes by increasing the salience of risk as a factor in decisions and that these shifts could exacerbate racial disparities. These results demonstrate that improving human prediction accuracy with algorithms does not necessarily improve human decisions and highlight the need to experimentally test how government algorithms are used by human decision-makers.

Related papers

• ABI Approach: Automatic Bias Identification in Decision-Making Under Risk based in an Ontology of Behavioral Economics [46.57327530703435]
  Risk seeking preferences for losses, driven by biases such as loss aversion, pose challenges and can result in severe negative consequences. This research introduces the ABI approach, a novel solution designed to support organizational decision-makers by automatically identifying and explaining risk seeking preferences.
  arXiv  Detail & Related papers  (2024-05-22T23:53:46Z)
• Reduced-Rank Multi-objective Policy Learning and Optimization [57.978477569678844]
  In practice, causal researchers do not have a single outcome in mind a priori. In government-assisted social benefit programs, policymakers collect many outcomes to understand the multidimensional nature of poverty. We present a data-driven dimensionality-reduction methodology for multiple outcomes in the context of optimal policy learning.
  arXiv  Detail & Related papers  (2024-04-29T08:16:30Z)
• Conformal Decision Theory: Safe Autonomous Decisions from Imperfect Predictions [80.34972679938483]
  We introduce Conformal Decision Theory, a framework for producing safe autonomous decisions despite imperfect machine learning predictions. Decisions produced by our algorithms are safe in the sense that they come with provable statistical guarantees of having low risk. Experiments demonstrate the utility of our approach in robot motion planning around humans, automated stock trading, and robot manufacturing.
  arXiv  Detail & Related papers  (2023-10-09T17:59:30Z)
• Bayesian Safe Policy Learning with Chance Constrained Optimization: Application to Military Security Assessment during the Vietnam War [0.0]
  We investigate whether it would have been possible to improve a security assessment algorithm employed during the Vietnam War. This empirical application raises several methodological challenges that frequently arise in high-stakes algorithmic decision-making.
  arXiv  Detail & Related papers  (2023-07-17T20:59:50Z)
• Influence of the algorithm's reliability and transparency in the user's decision-making process [0.0]
  We conduct an online empirical study with 61 participants to find out how the change in transparency and reliability of an algorithm could impact users' decision-making process. The results indicate that people show at least moderate confidence in the decisions of the algorithm even when the reliability is bad.
  arXiv  Detail & Related papers  (2023-07-13T03:13:49Z)
• Robust Design and Evaluation of Predictive Algorithms under Unobserved Confounding [2.8498944632323755]
  We propose a unified framework for the robust design and evaluation of predictive algorithms in selectively observed data. We impose general assumptions on how much the outcome may vary on average between unselected and selected units. We develop debiased machine learning estimators for the bounds on a large class of predictive performance estimands.
  arXiv  Detail & Related papers  (2022-12-19T20:41:44Z)
• Algorithmic Assistance with Recommendation-Dependent Preferences [2.864550757598007]
  We consider the effect and design of algorithmic recommendations when they affect choices. We show that recommendation-dependent preferences create inefficiencies where the decision-maker is overly responsive to the recommendation.
  arXiv  Detail & Related papers  (2022-08-16T09:24:47Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• The Impact of Algorithmic Risk Assessments on Human Predictions and its Analysis via Crowdsourcing Studies [79.66833203975729]
  We conduct a vignette study in which laypersons are tasked with predicting future re-arrests. Our key findings are as follows: Participants often predict that an offender will be rearrested even when they deem the likelihood of re-arrest to be well below 50%. Judicial decisions, unlike participants' predictions, depend in part on factors that are to the likelihood of re-arrest.
  arXiv  Detail & Related papers  (2021-09-03T11:09:10Z)
• Experimental Evaluation of Algorithm-Assisted Human Decision-Making: Application to Pretrial Public Safety Assessment [0.8749675983608171]
  We develop a statistical methodology for experimentally evaluating the causal impacts of algorithmic recommendations on human decisions. We apply the proposed methodology to preliminary data from the first-ever randomized controlled trial. We find that providing the PSA to the judge has little overall impact on the judge's decisions and subsequent arrestee behavior.
  arXiv  Detail & Related papers  (2020-12-04T20:48:44Z)
• A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores [85.12096045419686]
  We study the adoption of an algorithmic tool used to assist child maltreatment hotline screening decisions. We first show that humans do alter their behavior when the tool is deployed. We show that humans are less likely to adhere to the machine's recommendation when the score displayed is an incorrect estimate of risk.
  arXiv  Detail & Related papers  (2020-02-19T07:27:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.