Accuracy, Fairness, and Interpretability of Machine Learning Criminal Recidivism Models

• URL: http://arxiv.org/abs/2209.14237v1
• Date: Wed, 14 Sep 2022 17:53:24 GMT
• Title: Accuracy, Fairness, and Interpretability of Machine Learning Criminal Recidivism Models
• Authors: Eric Ingram, Furkan Gursoy, Ioannis A. Kakadiaris
• Abstract summary: Various machine learning-based criminal recidivism models are created based on a real-world parole decision dataset from the state of Georgia in the United States. It is found that there are noted differences and trade-offs between accuracy, fairness, and being inherently interpretable.
• Score: 4.297070083645049
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Criminal recidivism models are tools that have gained widespread adoption by parole boards across the United States to assist with parole decisions. These models take in large amounts of data about an individual and then predict whether an individual would commit a crime if released on parole. Although such models are not the only or primary factor in making the final parole decision, questions have been raised about their accuracy, fairness, and interpretability. In this paper, various machine learning-based criminal recidivism models are created based on a real-world parole decision dataset from the state of Georgia in the United States. The recidivism models are comparatively evaluated for their accuracy, fairness, and interpretability. It is found that there are noted differences and trade-offs between accuracy, fairness, and being inherently interpretable. Therefore, choosing the best model depends on the desired balance between accuracy, fairness, and interpretability, as no model is perfect or consistently the best across different criteria.

Related papers

• The Judge Variable: Challenging Judge-Agnostic Legal Judgment Prediction [0.0]
  This study examines the role of human judges in legal decision-making.<n>It uses machine learning to predict child physical custody outcomes in French appellate courts.
  arXiv  Detail & Related papers  (2025-07-18T08:28:53Z)
• Fact-or-Fair: A Checklist for Behavioral Testing of AI Models on Fairness-Related Queries [85.909363478929]
  In this study, we focus on 19 real-world statistics collected from authoritative sources. We develop a checklist comprising objective and subjective queries to analyze behavior of large language models. We propose metrics to assess factuality and fairness, and formally prove the inherent trade-off between these two aspects.
  arXiv  Detail & Related papers  (2025-02-09T10:54:11Z)
• Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Set [9.440660971920648]
  We study the properties of sets of equally accurate models, or Rashomon sets, in general. Our contributions include methods for efficiently sampling models from this set. We also derive the probability that an individual's prediction will be flipped within the Rashomon set.
  arXiv  Detail & Related papers  (2025-01-26T18:39:54Z)
• Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction [50.62245481416744]
  We present the first benchmark that simulates the evaluation of open information extraction models in the real world. We design and annotate a large-scale testbed in which each example is a knowledge-invariant clique. By further elaborating the robustness metric, a model is judged to be robust if its performance is consistently accurate on the overall cliques.
  arXiv  Detail & Related papers  (2023-05-23T12:05:09Z)
• DualFair: Fair Representation Learning at Both Group and Individual Levels via Contrastive Self-supervision [73.80009454050858]
  This work presents a self-supervised model, called DualFair, that can debias sensitive attributes like gender and race from learned representations. Our model jointly optimize for two fairness criteria - group fairness and counterfactual fairness.
  arXiv  Detail & Related papers  (2023-03-15T07:13:54Z)
• Revealing Unfair Models by Mining Interpretable Evidence [50.48264727620845]
  The popularity of machine learning has increased the risk of unfair models getting deployed in high-stake applications. In this paper, we tackle the novel task of revealing unfair models by mining interpretable evidence. Our method finds highly interpretable and solid evidence to effectively reveal the unfairness of trained models.
  arXiv  Detail & Related papers  (2022-07-12T20:03:08Z)
• Fairness-aware Summarization for Justified Decision-Making [16.47665757950391]
  We focus on the problem of (un)fairness in the justification of the text-based neural models. We propose a fairness-aware summarization mechanism to detect and counteract the bias in such models.
  arXiv  Detail & Related papers  (2021-07-13T17:04:10Z)
• To what extent do human explanations of model behavior align with actual model behavior? [91.67905128825402]
  We investigated the extent to which human-generated explanations of models' inference decisions align with how models actually make these decisions. We defined two alignment metrics that quantify how well natural language human explanations align with model sensitivity to input words. We find that a model's alignment with human explanations is not predicted by the model's accuracy on NLI.
  arXiv  Detail & Related papers  (2020-12-24T17:40:06Z)
• Impact of Accuracy on Model Interpretations [24.975981795360845]
  It is vital for a data scientist to choose trustworthy interpretations to drive real world impact. We propose two metrics to quantify the quality of an interpretation and design an experiment to test how these metrics vary with model accuracy. We find that for datasets that can be modeled accurately by a variety of methods, simpler methods yield higher quality interpretations.
  arXiv  Detail & Related papers  (2020-11-17T19:02:59Z)
• Fairness in Semi-supervised Learning: Unlabeled Data Help to Reduce Discrimination [53.3082498402884]
  A growing specter in the rise of machine learning is whether the decisions made by machine learning models are fair. We present a framework of fair semi-supervised learning in the pre-processing phase, including pseudo labeling to predict labels for unlabeled data. A theoretical decomposition analysis of bias, variance and noise highlights the different sources of discrimination and the impact they have on fairness in semi-supervised learning.
  arXiv  Detail & Related papers  (2020-09-25T05:48:56Z)
• Are Visual Explanations Useful? A Case Study in Model-in-the-Loop Prediction [49.254162397086006]
  We study explanations based on visual saliency in an image-based age prediction task. We find that presenting model predictions improves human accuracy. However, explanations of various kinds fail to significantly alter human accuracy or trust in the model.
  arXiv  Detail & Related papers  (2020-07-23T20:39:40Z)
• In Pursuit of Interpretable, Fair and Accurate Machine Learning for Criminal Recidivism Prediction [19.346391120556884]
  This study trains interpretable models that output probabilities rather than binary predictions, and uses quantitative fairness definitions to assess the models. We generated black-box and interpretable ML models on two different criminal recidivism datasets from Florida and Kentucky. Several interpretable ML models can predict recidivism as well as black-box ML models and are more accurate than COMPAS or the Arnold PSA.
  arXiv  Detail & Related papers  (2020-05-08T17:16:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.