(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
- URL: http://arxiv.org/abs/2403.13004v1
- Date: Wed, 13 Mar 2024 23:19:46 GMT
- Title: (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
- Authors: Angela Jin, Niloufar Salehi,
- Abstract summary: We study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court.
We present findings from interviews with 17 people in the U.S. public defense community.
- Score: 7.742399489996169
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (CFS) -- automated decision systems that the government uses to convict and incarcerate, such as facial recognition, gunshot detection, and probabilistic genotyping tools. We find that our participants faced challenges assessing and contesting CFS reliability due to difficulties (a) navigating how CFS is developed and used, (b) overcoming judges and jurors' non-critical perceptions of CFS, and (c) gathering CFS expertise. To conclude, we provide recommendations that center the technical, social, and institutional context to better position interventions such as performance evaluations to support contestability in practice.
Related papers
- Combining AI Control Systems and Human Decision Support via Robustness and Criticality [53.10194953873209]
We extend a methodology for adversarial explanations (AE) to state-of-the-art reinforcement learning frameworks.
We show that the learned AI control system demonstrates robustness against adversarial tampering.
In a training / learning framework, this technology can improve both the AI's decisions and explanations through human interaction.
arXiv Detail & Related papers (2024-07-03T15:38:57Z) - Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits [54.648819983899614]
Particip-AI is a framework to gather current and future AI use cases and their harms and benefits from non-expert public.
We gather responses from 295 demographically diverse participants.
arXiv Detail & Related papers (2024-03-21T19:12:37Z) - Recommendations for Government Development and Use of Advanced Automated
Systems to Make Decisions about Individuals [14.957989495850935]
Contestability is often constitutionally required as an element of due process.
We convened a workshop on advanced automated decision making, contestability, and the law.
arXiv Detail & Related papers (2024-03-04T00:03:00Z) - Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness [53.91018508439669]
The study explores the complexities of integrating Artificial Intelligence into Autonomous Vehicles (AVs)
It examines the challenges introduced by AI components and the impact on testing procedures.
The paper identifies significant challenges and suggests future directions for research and development of AI in AV technology.
arXiv Detail & Related papers (2024-02-21T08:29:42Z) - Equal Confusion Fairness: Measuring Group-Based Disparities in Automated
Decision Systems [5.076419064097733]
This paper proposes a new equal confusion fairness test to check an automated decision system for fairness and a new confusion parity error to quantify the extent of any unfairness.
Overall, the methods and metrics provided here may assess automated decision systems' fairness as part of a more extensive accountability assessment.
arXiv Detail & Related papers (2023-07-02T04:44:19Z) - Perspectives on Large Language Models for Relevance Judgment [56.935731584323996]
Large language models (LLMs) claim that they can assist with relevance judgments.
It is not clear whether automated judgments can reliably be used in evaluations of retrieval systems.
arXiv Detail & Related papers (2023-04-13T13:08:38Z) - Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax
Audit Models [73.24381010980606]
This study examines issues of algorithmic fairness in the context of systems that inform tax audit selection by the IRS.
We show how the use of more flexible machine learning methods for selecting audits may affect vertical equity.
Our results have implications for the design of algorithmic tools across the public sector.
arXiv Detail & Related papers (2022-06-20T16:27:06Z) - Adversarial Scrutiny of Evidentiary Statistical Software [32.962815960406196]
U.S. criminal legal system increasingly relies on software output to convict and incarcerate people.
We propose robust adversarial testing as an audit framework to examine the validity of evidentiary statistical software.
arXiv Detail & Related papers (2022-06-19T02:08:42Z) - Unfooling Perturbation-Based Post Hoc Explainers [12.599362066650842]
Recent work demonstrates that perturbation-based post hoc explainers can be fooled adversarially.
This discovery has adverse implications for auditors, regulators, and other sentinels.
In this work, we rigorously formalize this problem and devise a defense against adversarial attacks on perturbation-based explainers.
arXiv Detail & Related papers (2022-05-29T21:28:12Z) - Differential Privacy and Fairness in Decisions and Learning Tasks: A
Survey [50.90773979394264]
It reviews the conditions under which privacy and fairness may have aligned or contrasting goals.
It analyzes how and why DP may exacerbate bias and unfairness in decision problems and learning tasks.
arXiv Detail & Related papers (2022-02-16T16:50:23Z) - Performance in the Courtroom: Automated Processing and Visualization of
Appeal Court Decisions in France [20.745220428708457]
We use NLP methods to extract interesting entities/data from judgments to construct networks of lawyers and judgments.
We propose metrics to rank lawyers based on their experience, wins/loss ratio and their importance in the network of lawyers.
arXiv Detail & Related papers (2020-06-11T08:22:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.