Learning When Not to Decide: A Framework for Overcoming Factual Presumptuousness in AI Adjudication

• URL: http://arxiv.org/abs/2604.19895v1
• Date: Tue, 21 Apr 2026 18:17:08 GMT
• Title: Learning When Not to Decide: A Framework for Overcoming Factual Presumptuousness in AI Adjudication
• Abstract summary: A well-known limitation of AI systems is the tendency to provide confident answers when information may be lacking.<n>This challenge is particularly acute in legal applications, where a core task for attorneys, judges, and administrators is to determine whether evidence is sufficient to reach a conclusion.<n>We study this problem in the important setting of unemployment insurance adjudication, which has seen rapid integration of AI systems.
• Score: 4.991125406994611
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A well-known limitation of AI systems is presumptuousness: the tendency of AI systems to provide confident answers when information may be lacking. This challenge is particularly acute in legal applications, where a core task for attorneys, judges, and administrators is to determine whether evidence is sufficient to reach a conclusion. We study this problem in the important setting of unemployment insurance adjudication, which has seen rapid integration of AI systems and where the question of additional fact-finding poses the most significant bottleneck for a system that affects millions of applicants annually. First, through a collaboration with the Colorado Department of Labor and Employment, we secure rare access to official training materials and guidance to design a novel benchmark that systematically varies in information completeness. Second, we evaluate four leading AI platforms and show that standard RAG-based approaches achieve an average of only 15% accuracy when information is insufficient. Third, advanced prompting methods improve accuracy on inconclusive cases but over-correct, withholding decisions even on clear cases. Fourth, we introduce a structured framework requiring explicit identification of missing information before any determination (SPEC, Structured Prompting for Evidence Checklists). SPEC achieves 89% overall accuracy, while appropriately deferring when evidence is insufficient -- demonstrating that presumptuousness in legal AI is systematic but addressable, and that doing so is a necessary step towards systems that reliably support, rather than supplant, human judgment wherever decisions must await sufficient evidence.

Related papers

• Bridging the First-Hour Gap: Evaluating AI Reliability and Benchmarking Deficiencies in Cyber Incident Response for Law Enforcement [0.0]
  This paper provides a systematic survey of decision-support architectures designed to assist first responders of a cybercrime.<n>Our analysis identifies RAG-based systems as a relatively viable intermediate solution due to their natural language adaptability.<n>We argue for the necessity of a new evaluation benchmark focused on naive robustness query and evidence preservation.
  arXiv  Detail & Related papers  (2026-09-11T10:25:28Z)
• Validity, Reliability, and Transparency in Artificial Intelligence Regulation [2.588958866512272]
  Article argues that modern AI raises concerns of construct validity, confounding, representativeness, distribution shift, and fairness trade-offs.<n>We argue that inference should function as a precondition for proportionality assessment and deployment approval.
  arXiv  Detail & Related papers  (2026-08-06T09:36:15Z)
• AILQA: Evaluating AI-Driven Legal Question Answering Systems for the Indian Legal System [11.383388187292674]
  This study introduces an advanced Artificial Intelligence for Indian Legal Question Answering (AILQA) system tailored to the Indian legal context.<n>AILQA leverages a variety of embedding and generative models, including recent Large Language Models (LLMs)<n>We conducted rigorous evaluations using both lexical and semantic metrics, enriched by expert legal feedback, to ensure relevance and accuracy.
  arXiv  Detail & Related papers  (2026-07-21T08:01:17Z)
• FIKA-Bench: From Fine-grained Recognition to Fine-Grained Knowledge Acquisition [54.31138496553705]
  Fine-grained recognition in everyday life is often not a closed-book classification problem.<n>Existing benchmarks primarily evaluate visually recognition, leaving this active external knowledge acquisition ability underexplored.<n>We study fine-grained knowledge acquisition, where a system must seek, verify, and use external evidence to answer open-ended fine-grained recognition questions.
  arXiv  Detail & Related papers  (2026-05-13T08:49:51Z)
• Is your AI Model Accurate Enough? The Difficult Choices Behind Rigorous AI Development and the EU AI Act [11.9330079062021]
  The 2024 European Union AI Act mandates an "appropriate level of accuracy" for high-risk systems.<n>This paper provides a legal-technical analysis of the choices that shape how accuracy is defined, measured, and assessed.<n>By making the techno-normative dimensions of accuracy explicit, this paper contributes to broader interdisciplinary debates on AI governance and regulation.
  arXiv  Detail & Related papers  (2026-03-11T15:51:37Z)
• Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies [57.521647436515785]
  We define frontier AI auditing as rigorous third-party verification of frontier AI developers' safety and security claims.<n>We introduce AI Assurance Levels (AAL-1 to AAL-4), ranging from time-bounded system audits to continuous, deception-resilient verification.
  arXiv  Detail & Related papers  (2026-01-16T18:44:09Z)
• Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions [51.56484100374058]
  Current large language models (LLMs) excel in verifiable domains where outputs can be checked before action but prove less reliable for high-stakes strategic decisions with uncertain outcomes.<n>This gap, driven by mutually cognitive biases in both humans and artificial intelligence (AI) systems, threatens the defensibility of valuations and sustainability of investments in the sector.<n>This report describes a framework emerging from systematic qualitative assessment across 7 frontier-grade LLMs and 3 market-facing venture vignettes under time pressure.
  arXiv  Detail & Related papers  (2025-11-10T22:24:21Z)
• Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metrics [89.1999907891494]
  We present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox.<n>Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures.<n>We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies.
  arXiv  Detail & Related papers  (2025-10-01T07:59:03Z)
• Judicial Requirements for Generative AI in Legal Reasoning [0.0]
  Large Language Models (LLMs) are being integrated into professional domains, yet their limitations in high-stakes fields like law remain poorly understood.<n>This paper defines the core capabilities that an AI system must possess to function as a reliable reasoning tool in judicial decision-making.
  arXiv  Detail & Related papers  (2025-08-26T09:56:26Z)
• Towards Robust Fact-Checking: A Multi-Agent System with Advanced Evidence Retrieval [1.515687944002438]
  The rapid spread of misinformation in the digital era poses significant challenges to public discourse.<n>Traditional human-led fact-checking methods, while credible, struggle with the volume and velocity of online content.<n>This paper proposes a novel multi-agent system for automated fact-checking that enhances accuracy, efficiency, and explainability.
  arXiv  Detail & Related papers  (2025-06-22T02:39:27Z)
• Towards User-Centred Design of AI-Assisted Decision-Making in Law Enforcement [1.1890528509539204]
  User requirements for AI-assisted systems in law enforcement remain unclear.<n>Participants in our study highlighted the need for a system capable of processing and analysing large volumes of data efficiently.<n>We argue that it is very unlikely that the system will ever achieve full automation due to the dynamic and complex nature of the law enforcement domain.
  arXiv  Detail & Related papers  (2025-04-24T09:25:29Z)
• AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons [62.374792825813394]
  This paper introduces AILuminate v1.0, the first comprehensive industry-standard benchmark for assessing AI-product risk and reliability.<n>The benchmark evaluates an AI system's resistance to prompts designed to elicit dangerous, illegal, or undesirable behavior in 12 hazard categories.
  arXiv  Detail & Related papers  (2025-02-19T05:58:52Z)
• Trustworthy AI [75.99046162669997]
  Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, are some of the most prominent limitations. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems.
  arXiv  Detail & Related papers  (2020-11-02T20:04:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.