The Progression of Disparities within the Criminal Justice System: Differential Enforcement and Risk Assessment Instruments

• URL: http://arxiv.org/abs/2305.07575v1
• Date: Fri, 12 May 2023 16:06:40 GMT
• Title: The Progression of Disparities within the Criminal Justice System: Differential Enforcement and Risk Assessment Instruments
• Abstract summary: Algorithmic risk assessment instruments (RAIs) increasingly inform decision-making in criminal justice. Problematically, the extent to which arrests reflect overall offending can vary with the person's characteristics. We examine how the disconnect between crime and arrest rates impacts RAIs and their evaluation.
• Score: 26.018802058292614
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Algorithmic risk assessment instruments (RAIs) increasingly inform decision-making in criminal justice. RAIs largely rely on arrest records as a proxy for underlying crime. Problematically, the extent to which arrests reflect overall offending can vary with the person's characteristics. We examine how the disconnect between crime and arrest rates impacts RAIs and their evaluation. Our main contribution is a method for quantifying this bias via estimation of the amount of unobserved offenses associated with particular demographics. These unobserved offenses are then used to augment real-world arrest records to create part real, part synthetic crime records. Using this data, we estimate that four currently deployed RAIs assign 0.5--2.8 percentage points higher risk scores to Black individuals than to White individuals with a similar \emph{arrest} record, but the gap grows to 4.5--11.0 percentage points when we match on the semi-synthetic \emph{crime} record. We conclude by discussing the potential risks around the use of RAIs, highlighting how they may exacerbate existing inequalities if the underlying disparities of the criminal justice system are not taken into account. In light of our findings, we provide recommendations to improve the development and evaluation of such tools.

Related papers

• The Likelihood Ratio Wall: Structural Limits on Accurate Risk Assessment for Rare Violence [0.0]
  Pretrial risk assessment tools are used on over one million U.S. defendants each year.<n>We show that their use for predicting rare violent re-offense faces a basic statistical barrier.
  arXiv  Detail & Related papers  (2026-04-30T00:32:52Z)
• Unmasking Algorithmic Bias in Predictive Policing: A GAN-Based Simulation Framework with Multi-City Temporal Analysis [0.0]
  We show how racial bias propagates through the full enforcement pipeline from crime occurrence to police contact.<n>Experiments reveal extreme and year variant bias in Baltimores detected mode, with mean annual DIR up to 15714 in 2019, moderate under detection of Black residents in Chicago DIR equals 0.22.<n>We further demonstrate that a Conditional Tabular GAN CTGAN debiasing approach partially redistributes detection rates but cannot eliminate structural disparity without accompanying policy intervention.
  arXiv  Detail & Related papers  (2026-03-19T14:55:32Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts [54.15982476754607]
  Large language models (LLMs) are now deployed at unprecedented scale, assisting millions of users in daily tasks.<n>This study defines complicit facilitation as the provision of guidance or support that enables illicit user instructions.<n>Using real-world legal cases and established legal frameworks, we construct an evaluation benchmark spanning 269 illicit scenarios and 50 illicit intents.
  arXiv  Detail & Related papers  (2025-11-25T16:01:31Z)
• Partial Identification Approach to Counterfactual Fairness Assessment [50.88100567472179]
  We introduce a Bayesian approach to bound unknown counterfactual fairness measures with high confidence.<n>Our results reveal a positive (spurious) effect on the COMPAS score when changing race to African-American (from all others) and a negative (direct causal) effect when transitioning from young to old age.
  arXiv  Detail & Related papers  (2025-09-30T18:35:08Z)
• Fairness Is More Than Algorithms: Racial Disparities in Time-to-Recidivism [14.402936852692408]
  This work introduces the notion of counterfactual racial disparity and offers a formal test using observational data to assess if differences in recidivism arise from algorithmic bias, contextual factors, or their interplay. An empirical study applying this framework to the COMPAS dataset reveals that short-term recidivism patterns do not exhibit racial disparities when controlling for risk scores. This suggests that factors beyond algorithmic scores, possibly structural disparities in housing, employment, and social support, may accumulate and exacerbate recidivism risks over time.
  arXiv  Detail & Related papers  (2025-04-25T18:13:37Z)
• Criticality and Safety Margins for Reinforcement Learning [53.10194953873209]
  We seek to define a criticality framework with both a quantifiable ground truth and a clear significance to users. We introduce true criticality as the expected drop in reward when an agent deviates from its policy for n consecutive random actions. We also introduce the concept of proxy criticality, a low-overhead metric that has a statistically monotonic relationship to true criticality.
  arXiv  Detail & Related papers  (2024-09-26T21:00:45Z)
• Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition [56.69875958980474]
  This work considers approaches to defending learned systems and how security defenses result in performance inequities across different sub-populations. We find that many methods that have been proposed can cause direct harm, like false rejection and unequal benefits from robustness training. We present a comparison of equality between two rejection-based defenses: randomized smoothing and neural rejection, finding randomized smoothing more equitable due to the sampling mechanism for minority groups.
  arXiv  Detail & Related papers  (2023-02-17T16:19:26Z)
• Flipping the Script on Criminal Justice Risk Assessment: An actuarial model for assessing the risk the federal sentencing system poses to defendants [0.0]
  algorithmic risk assessment instruments are used to predict the risk a defendant poses to society. We develop a risk assessment instrument that "flips the script" Our instrument achieves comparable predictive accuracy to risk assessment instruments used in pretrial and parole contexts.
  arXiv  Detail & Related papers  (2022-05-26T17:17:13Z)
• Spatial-Temporal Hypergraph Self-Supervised Learning for Crime Prediction [60.508960752148454]
  This work proposes a Spatial-Temporal Hypergraph Self-Supervised Learning framework to tackle the label scarcity issue in crime prediction. We propose the cross-region hypergraph structure learning to encode region-wise crime dependency under the entire urban space. We also design the dual-stage self-supervised learning paradigm, to not only jointly capture local- and global-level spatial-temporal crime patterns, but also supplement the sparse crime representation by augmenting region self-discrimination.
  arXiv  Detail & Related papers  (2022-04-18T23:46:01Z)
• Deep Interpretable Criminal Charge Prediction and Algorithmic Bias [2.3347476425292717]
  This paper addresses bias issues with post-hoc explanations to provide a trustable prediction of whether a person will receive future criminal charges. Our approach shows consistent and reliable prediction precision and recall on a real-life dataset.
  arXiv  Detail & Related papers  (2021-06-25T07:00:13Z)
• The effect of differential victim crime reporting on predictive policing systems [84.86615754515252]
  We show how differential victim crime reporting rates can lead to outcome disparities in common crime hot spot prediction models. Our results suggest that differential crime reporting rates can lead to a displacement of predicted hotspots from high crime but low reporting areas to high or medium crime and high reporting areas.
  arXiv  Detail & Related papers  (2021-01-30T01:57:22Z)
• Improving Fairness in Criminal Justice Algorithmic Risk Assessments Using Conformal Prediction Sets [0.0]
  We adopt a framework from conformal prediction sets to remove unfairness from risk algorithms. From a sample of 300,000 offenders at their arraignments, we construct a confusion table and its derived measures of fairness. We see our work as a demonstration of concept for application in a wide variety of criminal justice decisions.
  arXiv  Detail & Related papers  (2020-08-26T16:47:02Z)
• Fairness Evaluation in Presence of Biased Noisy Labels [84.12514975093826]
  We propose a sensitivity analysis framework for assessing how assumptions on the noise across groups affect the predictive bias properties of the risk assessment model. Our experimental results on two real world criminal justice data sets demonstrate how even small biases in the observed labels may call into question the conclusions of an analysis based on the noisy outcome.
  arXiv  Detail & Related papers  (2020-03-30T20:47:00Z)
• Crime Prediction Using Spatio-Temporal Data [8.50468505606714]
  Supervised learning technique is used to predict crimes with better accuracy. The proposed system is feed with a criminal-activity data set of twelve years of San Francisco city.
  arXiv  Detail & Related papers  (2020-03-11T16:19:19Z)
• Exploring Spatio-Temporal and Cross-Type Correlations for Crime Prediction [48.1813701535167]
  We perform crime prediction exploiting the cross-type and-temporal correlations of urban crimes. We propose a coherent framework to mathematically model these correlations for crime prediction. Further experiments have been conducted to understand the importance of different correlations in crime prediction.
  arXiv  Detail & Related papers  (2020-01-20T00:34:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.