From Process to Evidence: How Computing Can Ground Appropriate Reliance on Legal AI

• URL: http://arxiv.org/abs/2607.28869v1
• Date: Thu, 30 Jul 2026 22:24:59 GMT
• Title: From Process to Evidence: How Computing Can Ground Appropriate Reliance on Legal AI
• Abstract summary: After more than 1,500 cases involving AI hallucinations, lawyers have been instructed to perform careful, independent review of AI-assisted filings.<n>Discharging these duties requires what the human-computer interaction literature calls appropriate reliance''<n>We analyze the official record of the New York court system.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Lawyers and self-represented litigants are already using artificial intelligence (AI) to draft legal documents, and courts are responding with rules. After more than 1,500 cases involving AI hallucinations, lawyers have been instructed to perform careful, independent review of AI-assisted filings. Discharging these duties requires what the human-computer interaction (HCI) literature calls ``appropriate reliance,'' which cannot be calibrated without evidence on how often, how badly, and how detectably these tools fail at legal work. Existing research barely describes any of the three. We analyze the official record of the New York court system. The documents repeatedly call for evidence that does not exist (e.g., error rates, do-not-use lists). In its place they invoke procedure, including training mandates, checklists, and uncalibrated human review. The burden falls hardest on those least equipped to bear it: legal aid programs are told to track their own error rates, and judges are left to improvise their own tests. The paper makes four contributions: (1) a mapping from the legal duties to concepts in HCI; (2) a set of requirements elicited from the official record; (3) an analysis of how the legal system substitutes process for evidence; and (4) a research agenda for computing, including task taxonomies, shared error metrics, maintained benchmarks, and test harnesses for evaluations on private data. The computing community must supply what the justice system lacks; in doing so, it can help close, rather than widen, the justice gap.

Related papers

• Credible, Not Always Correct: How Reddit Users Verify AI-Generated Legal Advice [0.46976113832881716]
  Large language models (LLMs) are increasingly used by laypeople to resolve real legal problems.<n>This article presents evidence that the practical force of AI-generated legal advice depends not on its accuracy but on the social production of its credibility.
  arXiv  Detail & Related papers  (2026-08-13T15:33:12Z)
• What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Education [3.2940515388832963]
  This paper develops a framework for AI-mediated assessment that links the learning claim, delegation boundary, evidence standard, and safeguards.<n>We then audit verified public generative AI assessment guidance from 30 universities.
  arXiv  Detail & Related papers  (2026-07-22T10:21:16Z)
• Who Checks the Citations? Benchmarking Legal Hallucination Detection [7.592193069681834]
  Attorneys, judges, and pro se filers increasingly use AI to draft legal documents, yet these tools frequently fabricate citations.<n>This study evaluates whether AI-based systems can mitigate these errors by automatically detecting hallucinations.<n>We propose a taxonomy of legal citation hallucinations grounded in actual court filings and introduce a dataset of 1,300 brief excerpts containing injected errors.
  arXiv  Detail & Related papers  (2026-06-19T06:47:31Z)
• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)
• LawThinker: A Deep Research Legal Agent in Dynamic Environments [51.782293183431676]
  LawThinker is an autonomous legal research agent.<n>It enforces verification as an atomic operation after every knowledge exploration step.<n>LawThinker achieves a 24% improvement over direct reasoning.
  arXiv  Detail & Related papers  (2026-02-12T15:19:11Z)
• How Can AI Augment Access to Justice? Public Defenders' Perspectives on AI Adoption [6.949832807566659]
  We find that AI adoption is constrained by costs, restrictive office norms, confidentiality risks, and unsatisfactory tool quality.<n>Public defenders view AI as most useful for evidence investigation to analyze overwhelming amounts of digital records.<n>Courtroom representation and defense strategy are considered least compatible with AI assistance.
  arXiv  Detail & Related papers  (2025-10-27T02:26:08Z)
• Tasks and Roles in Legal AI: Data Curation, Annotation, and Verification [4.099848175176399]
  The application of AI tools to the legal field feels natural.<n>However, legal documents differ from the web-based text that underlies most AI systems.<n>We identify three areas of special relevance to practitioners: data curation, data annotation, and output verification.
  arXiv  Detail & Related papers  (2025-04-02T04:34:58Z)
• AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction [56.797874973414636]
  AnnoCaseLaw is a first-of-its-kind dataset of 471 meticulously annotated U.S. Appeals Court negligence cases.<n>Our dataset lays the groundwork for more human-aligned, explainable Legal Judgment Prediction models.<n>Results demonstrate that LJP remains a formidable task, with application of legal precedent proving particularly difficult.
  arXiv  Detail & Related papers  (2025-02-28T19:14:48Z)
• Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools [32.78336381381673]
  We report on the first preregistered empirical evaluation of AI-driven legal research tools. We find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. It provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs.
  arXiv  Detail & Related papers  (2024-05-30T17:56:05Z)
• The Ethics of Automating Legal Actors [58.81546227716182]
  We argue that automating the role of the judge raises difficult ethical challenges, in particular for common law legal systems. Our argument follows from the social role of the judge in actively shaping the law, rather than merely applying it. Even in the case the models could achieve human-level capabilities, there would still be remaining ethical concerns inherent in the automation of the legal process.
  arXiv  Detail & Related papers  (2023-12-01T13:48:46Z)
• SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval [75.05173891207214]
  Legal case retrieval plays a core role in the intelligent legal system. Most existing language models have difficulty understanding the long-distance dependencies between different structures. We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
  arXiv  Detail & Related papers  (2023-04-22T10:47:01Z)
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence [81.04070052740596]
  Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain. This paper introduces the history, the current state, and the future directions of research in LegalAI.
  arXiv  Detail & Related papers  (2020-04-25T14:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.