Confronting Label Indeterminacy in Automated Bail Decisions

• URL: http://arxiv.org/abs/2605.04073v1
• Date: Mon, 13 Apr 2026 15:23:12 GMT
• Title: Confronting Label Indeterminacy in Automated Bail Decisions
• Abstract summary: Bail decisions present a fundamental challenge for data-driven decision support systems.<n>When bail is denied, the counterfactual outcome of whether the defendant would have appeared in court remains unobserved.<n>We evaluate five contemporary approaches to handling label indeterminacy across three machine learning models.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bail decisions present a fundamental challenge for data-driven decision support systems. When bail is denied, the counterfactual outcome of whether the defendant would have appeared in court remains unobserved. As a result, historical bail data embed structural label indeterminacy: future decisions are influenced by past decisions whose outcomes are only partially knowable. Building automated systems on such data risks introducing bias and reinforcing feedback loops. This raises a core question for machine-learning systems intended to assist judicial actors: how should cases in which bail was denied be treated during model development? In a case study of bail decisions from the Unified Judicial System of Pennsylvania, we evaluate five contemporary approaches to handling label indeterminacy across three machine learning models, including a novel label imputation method motivated by the dynamics of bail decisions. Each method relies on unverifiable assumptions, yet all influence the models' predictive behaviour, sometimes even more so than the choice of model itself. Explainable AI analysis further reveals that these effects extend to the models' internal decision-making processes as well. Finally, we consider the notion of label indeterminacy from a legal perspective and assess the legitimacy of these approaches in the context of bail decision-making.

Related papers

• Explainable AI Isn't Enough! Rethinking Algorithmic Contestability [3.135152720206845]
  We propose an operational definition of contestability as a natural complement to recourse.<n>We identify three types of evidence warranting reversal according to the decision maker's own ethical standards.<n>We argue that these render decisions normatively indefensible and thus successfully contestable.
  arXiv  Detail & Related papers  (2026-05-15T15:14:13Z)
• Causal Algorithmic Recourse: Foundations and Methods [57.73269033551628]
  Key feature of AI systems is the ability to provide recommendations for how an individual may reverse a negative decision.<n>Existing approaches treat recourse outcomes as counterfactuals of a fixed unit.<n>We develop a causal framework that models recourse as a process over pre- and post-intervention outcomes.
  arXiv  Detail & Related papers  (2026-05-12T00:55:19Z)
• Fairness under uncertainty in sequential decisions [14.392931077157906]
  Algorithmic approaches alone cannot resolve structural inequalities.<n>Many real machine learning applications are online and sequential, with prior decisions informing future ones.<n>This paper introduces a taxonomy of uncertainty in sequential decision-making.
  arXiv  Detail & Related papers  (2026-04-23T14:17:58Z)
• Partial Identification Approach to Counterfactual Fairness Assessment [50.88100567472179]
  We introduce a Bayesian approach to bound unknown counterfactual fairness measures with high confidence.<n>Our results reveal a positive (spurious) effect on the COMPAS score when changing race to African-American (from all others) and a negative (direct causal) effect when transitioning from young to old age.
  arXiv  Detail & Related papers  (2025-09-30T18:35:08Z)
• Online Decision Mediation [72.80902932543474]
  Consider learning a decision support assistant to serve as an intermediary between (oracle) expert behavior and (imperfect) human behavior. In clinical diagnosis, fully-autonomous machine behavior is often beyond ethical affordances.
  arXiv  Detail & Related papers  (2023-10-28T05:59:43Z)
• One Model Many Scores: Using Multiverse Analysis to Prevent Fairness Hacking and Evaluate the Influence of Model Design Decisions [4.362723406385396]
  We show how multiverse analysis can be used to better understand fairness implications of design and evaluation decisions. Our results highlight how decisions regarding the evaluation of a system can lead to vastly different fairness metrics for the same model.
  arXiv  Detail & Related papers  (2023-08-31T12:32:43Z)
• Causal Fairness Analysis [68.12191782657437]
  We introduce a framework for understanding, modeling, and possibly solving issues of fairness in decision-making settings. The main insight of our approach will be to link the quantification of the disparities present on the observed data with the underlying, and often unobserved, collection of causal mechanisms. Our effort culminates in the Fairness Map, which is the first systematic attempt to organize and explain the relationship between different criteria found in the literature.
  arXiv  Detail & Related papers  (2022-07-23T01:06:34Z)
• Inverse Online Learning: Understanding Non-Stationary and Reactionary Policies [79.60322329952453]
  We show how to develop interpretable representations of how agents make decisions. By understanding the decision-making processes underlying a set of observed trajectories, we cast the policy inference problem as the inverse to this online learning problem. We introduce a practical algorithm for retrospectively estimating such perceived effects, alongside the process through which agents update them. Through application to the analysis of UNOS organ donation acceptance decisions, we demonstrate that our approach can bring valuable insights into the factors that govern decision processes and how they change over time.
  arXiv  Detail & Related papers  (2022-03-14T17:40:42Z)
• Leveraging Expert Consistency to Improve Algorithmic Decision Support [62.61153549123407]
  We explore the use of historical expert decisions as a rich source of information that can be combined with observed outcomes to narrow the construct gap. We propose an influence function-based methodology to estimate expert consistency indirectly when each case in the data is assessed by a single expert. Our empirical evaluation, using simulations in a clinical setting and real-world data from the child welfare domain, indicates that the proposed approach successfully narrows the construct gap.
  arXiv  Detail & Related papers  (2021-01-24T05:40:29Z)
• Predicting Court Decisions for Alimony: Avoiding Extra-legal Factors in Decision made by Judges and Not Understandable AI Models [0.02578242050187029]
  We present an explainable AI model designed in this purpose by combining a classification with random forest and a regression model. By using a large amount of court decisions in matters of divorce produced by French jurisdictions, we seek to identify if there may be extra-legal factors in the decisions taken by the judges.
  arXiv  Detail & Related papers  (2020-07-09T14:14:20Z)
• Inverse Active Sensing: Modeling and Understanding Timely Decision-Making [111.07204912245841]
  We develop a framework for the general setting of evidence-based decision-making under endogenous, context-dependent time pressure. We demonstrate how it enables modeling intuitive notions of surprise, suspense, and optimality in decision strategies.
  arXiv  Detail & Related papers  (2020-06-25T02:30:45Z)
• Modeling Perception Errors towards Robust Decision Making in Autonomous Vehicles [11.503090828741191]
  We propose a simulation-based methodology towards answering the question: is a perception subsystem sufficient for the decision making subsystem to make robust, safe decisions? We show how to analyze the impact of different kinds of sensing and perception errors on the behavior of the autonomous system.
  arXiv  Detail & Related papers  (2020-01-31T08:02:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.