Automating Transparency Mechanisms in the Judicial System Using LLMs: Opportunities and Challenges

• URL: http://arxiv.org/abs/2408.08477v1
• Date: Fri, 16 Aug 2024 01:40:23 GMT
• Title: Automating Transparency Mechanisms in the Judicial System Using LLMs: Opportunities and Challenges
• Abstract summary: LLMs have the potential to automate and scale these transparency pipelines. We discuss the opportunities and challenges of using LLMs to provide transparency in two important court processes.
• Score: 2.132063902944522
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bringing more transparency to the judicial system for the purposes of increasing accountability often demands extensive effort from auditors who must meticulously sift through numerous disorganized legal case files to detect patterns of bias and errors. For example, the high-profile investigation into the Curtis Flowers case took seven reporters a full year to assemble evidence about the prosecutor's history of selecting racially biased juries. LLMs have the potential to automate and scale these transparency pipelines, especially given their demonstrated capabilities to extract information from unstructured documents. We discuss the opportunities and challenges of using LLMs to provide transparency in two important court processes: jury selection in criminal trials and housing eviction cases.

Related papers

• Shadows of Transparency: Signaling Transparency-Impeding Behavior Using Public Data [0.0]
  Freedom of Information (FOI) laws aim to increase government transparency.<n>Existing assessments focus mainly on legal compliance and procedural outcomes, leaving organizational behavior underexamined.<n>This paper develops and evaluates a pattern-based approach that uses public response data to signal transparency-impeding behavior.
  arXiv  Detail & Related papers  (2026-06-12T12:09:57Z)
• CyberJurors: A Multi-Agent Simulation Task for E-Commerce Disputes Verdict [59.06704011432714]
  E-commerce platforms have begun recruiting crowdsourced jurors to adjudicate massive volumes of transaction disputes.<n>We introduce a pioneering task, E-commerce Dispute Verdicts (EDV), and present VerdictBench, a multimodal benchmark comprising 6,000 real-world cases.<n>We propose CyberJurors, a multi-agent framework to clarify the dispute logic and regulate the verdict process.
  arXiv  Detail & Related papers  (2026-05-27T12:07:44Z)
• Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks [123.36265437655187]
  Copyright Detective is an interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs.<n>It integrates multiple detection paradigms, including content recall testing, paraphrase-level similarity analysis, persuasive probing, and unlearning verification.
  arXiv  Detail & Related papers  (2026-02-05T03:09:52Z)
• Audit Trails for Accountability in Large Language Models [3.750249890675081]
  Large language models (LLMs) are increasingly embedded in consequential decisions across healthcare, finance, employment, and public services.<n>We propose audit trails as a sociotechnical mechanism for continuous accountability.<n>An audit trail is a chronological, tamper-evident, context-rich ledger of lifecycle events and decisions.
  arXiv  Detail & Related papers  (2026-01-28T16:04:33Z)
• ASP2LJ : An Adversarial Self-Play Laywer Augmented Legal Judgment Framework [21.003203706712643]
  Legal Judgment Prediction (LJP) aims to predict judicial outcomes, including relevant legal charge, terms, and fines.<n>Current datasets, derived from authentic cases, suffer from high human annotation costs and imbalanced distributions.<n>We propose an Adversarial Self-Play Lawyer Augmented Legal Judgment Framework, called ASP2LJ.<n>Our framework enables a judge to reference evolved lawyers' arguments, improving the objectivity, fairness, and rationality of judicial decisions.
  arXiv  Detail & Related papers  (2025-06-11T06:55:40Z)
• A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences [76.73731245899454]
  We propose a transparent law reasoning schema enriched with hierarchical factum probandum, evidence, and implicit experience. Inspired by this schema, we introduce the challenging task, which takes a textual case description and outputs a hierarchical structure justifying the final decision. This benchmark paves the way for transparent and accountable AI-assisted law reasoning in the Intelligent Court''
  arXiv  Detail & Related papers  (2025-03-02T10:26:54Z)
• AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Prediction [56.797874973414636]
  AnnoCaseLaw is a first-of-its-kind dataset of 471 meticulously annotated U.S. Appeals Court negligence cases. Our dataset lays the groundwork for more human-aligned, explainable Legal Judgment Prediction models. Results demonstrate that LJP remains a formidable task, with application of legal precedent proving particularly difficult.
  arXiv  Detail & Related papers  (2025-02-28T19:14:48Z)
• Auto-Drafting Police Reports from Noisy ASR Outputs: A Trust-Centered LLM Approach [11.469965123352287]
  This study presents an AI-driven system designed to generate police report drafts from complex, noisy, and multi-role dialogue data. Our approach intelligently extracts key elements of law enforcement interactions and includes them in the draft. This framework holds the potential to transform the reporting process, ensuring greater oversight, consistency, and fairness in future policing practices.
  arXiv  Detail & Related papers  (2025-02-11T16:27:28Z)
• LLMJudge: LLMs for Relevance Judgments [37.103230004631996]
  The challenge is organized as part of the LLM4Eval workshop at SIGIR 2024. Recent studies have shown that LLMs can generate reliable relevance judgments for search systems. The collected data will be released as a package to support automatic relevance judgment research.
  arXiv  Detail & Related papers  (2024-08-09T23:15:41Z)
• Learning Interpretable Legal Case Retrieval via Knowledge-Guided Case Reformulation [22.85652668826498]
  This paper introduces KELLER, a legal knowledge-guided case reformulation approach based on large language models (LLMs) By incorporating professional legal knowledge about crimes and law articles, we enable large language models to accurately reformulate the original legal case into concise sub-facts of crimes.
  arXiv  Detail & Related papers  (2024-06-28T08:59:45Z)
• CrAM: Credibility-Aware Attention Modification in LLMs for Combating Misinformation in RAG [50.030526904378256]
  Retrieval-Augmented Generation (RAG) can alleviate hallucinations of Large Language Models (LLMs) by referencing external documents. To address this issue, we explore the task of "credibility-aware RAG" We introduce a plug-and-play method named $textbfCr$edibility-aware $textbfA$ttention $textbfM$odification (CrAM) CrAM identifies influential attention heads and adjusts their attention weights based on the credibility of the documents, thereby reducing the impact of low-credibility documents.
  arXiv  Detail & Related papers  (2024-06-17T13:01:12Z)
• Automatic Information Extraction From Employment Tribunal Judgements Using Large Language Models [0.4810407297181484]
  This paper presents a study on the application of GPT-4, a large language model, for automatic information extraction from UK employment tribunal cases. We meticulously evaluated GPT-4's performance in extracting critical information with a manual verification process.
  arXiv  Detail & Related papers  (2024-03-19T17:43:08Z)
• Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning [49.23103067844278]
  We propose the task of multi-defendant LJP, which aims to automatically predict the judgment results for each defendant of multi-defendant cases. Two challenges arise with the task of multi-defendant LJP: (1) indistinguishable judgment results among various defendants; and (2) the lack of a real-world dataset for training and evaluation.
  arXiv  Detail & Related papers  (2023-12-10T04:46:30Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)
• Leveraging Large Language Models for Topic Classification in the Domain of Public Affairs [65.9077733300329]
  Large Language Models (LLMs) have the potential to greatly enhance the analysis of public affairs documents. LLMs can be of great use to process domain-specific documents, such as those in the domain of public affairs.
  arXiv  Detail & Related papers  (2023-06-05T13:35:01Z)
• FAT Forensics: A Python Toolbox for Implementing and Deploying Fairness, Accountability and Transparency Algorithms in Predictive Systems [69.24490096929709]
  We developed an open source Python package called FAT Forensics. It can inspect important fairness, accountability and transparency aspects of predictive algorithms. Our toolbox can evaluate all elements of a predictive pipeline.
  arXiv  Detail & Related papers  (2022-09-08T13:25:02Z)
• Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning in the Real Court Setting [25.53133777558123]
  We introduce a novel dataset from real courtrooms to predict the legal judgment in a reasonably encyclopedic manner. An extensive set of experiments with a large civil trial data set shows that the proposed model can more accurately characterize the interactions among claims, fact and debate for legal judgment prediction.
  arXiv  Detail & Related papers  (2021-07-12T04:27:14Z)
• Dimensions of Transparency in NLP Applications [64.16277166331298]
  Broader transparency in descriptions of and communication regarding AI systems is widely considered desirable. Previous work has suggested that a trade-off exists between greater system transparency and user confusion.
  arXiv  Detail & Related papers  (2021-01-02T11:46:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.