AI for Statutory Simplification: A Comprehensive State Legal Corpus and Labor Benchmark

• URL: http://arxiv.org/abs/2508.19365v1
• Date: Tue, 26 Aug 2025 18:53:39 GMT
• Title: AI for Statutory Simplification: A Comprehensive State Legal Corpus and Labor Benchmark
• Authors: Emaan Hariri, Daniel E. Ho,
• Abstract summary: One U.S. state has claimed to eliminate one third of its state code using AI.<n>We introduce LaborBench, a benchmark dataset to evaluate AI capabilities in this domain.
• Score: 5.268588811689132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the emerging use cases of AI in law is for code simplification: streamlining, distilling, and simplifying complex statutory or regulatory language. One U.S. state has claimed to eliminate one third of its state code using AI. Yet we lack systematic evaluations of the accuracy, reliability, and risks of such approaches. We introduce LaborBench, a question-and-answer benchmark dataset designed to evaluate AI capabilities in this domain. We leverage a unique data source to create LaborBench: a dataset updated annually by teams of lawyers at the U.S. Department of Labor, who compile differences in unemployment insurance laws across 50 states for over 101 dimensions in a six-month process, culminating in a 200-page publication of tables. Inspired by our collaboration with one U.S. state to explore using large language models (LLMs) to simplify codes in this domain, where complexity is particularly acute, we transform the DOL publication into LaborBench. This provides a unique benchmark for AI capacity to conduct, distill, and extract realistic statutory and regulatory information. To assess the performance of retrieval augmented generation (RAG) approaches, we also compile StateCodes, a novel and comprehensive state statute and regulatory corpus of 8.7 GB, enabling much more systematic research into state codes. We then benchmark the performance of information retrieval and state-of-the-art large LLMs on this data and show that while these models are helpful as preliminary research for code simplification, the overall accuracy is far below the touted promises for LLMs as end-to-end pipelines for regulatory simplification.

Related papers

• LegalOne: A Family of Foundation Models for Reliable Legal Reasoning [54.57434222018289]
  We present LegalOne, a family of foundational models specifically tailored for the Chinese legal domain.<n>LegalOne is developed through a comprehensive three-phase pipeline designed to master legal reasoning.<n>We publicly release the LegalOne weights and the LegalKit evaluation framework to advance the field of Legal AI.
  arXiv  Detail & Related papers  (2026-01-31T10:18:32Z)
• From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence [150.3696990310269]
  Large language models (LLMs) have transformed automated software development by enabling direct translation of natural language descriptions into functional code.<n>We provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs.<n>We analyze the code capability of the general LLMs (GPT-4, Claude, LLaMA) and code-specialized LLMs (StarCoder, Code LLaMA, DeepSeek-Coder, and QwenCoder)
  arXiv  Detail & Related papers  (2025-11-23T17:09:34Z)
• AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliance [10.49637840194233]
  There is growing interest in using Large Language Models (LLMs) to assess whether an AI system complies with a given AI Regulation (AIR)<n>We introduce AIReg-Bench: the first benchmark dataset designed to test how well LLMs can assess compliance with the EU AI Act (AIA)
  arXiv  Detail & Related papers  (2025-10-01T21:33:33Z)
• Scaling Legal AI: Benchmarking Mamba and Transformers for Statutory Classification and Case Law Retrieval [0.0]
  We present the first comprehensive benchmarking of Mamba, a state-space model with linear-time selective mechanisms, against leading transformer models for statutory classification and case law retrieval.<n>Results show that Mamba's linear scaling enables processing of legal documents several times longer than transformers.<n>Our findings highlight trade-offs between state-space models and transformers, providing guidance for deploying legal AI in statutory analysis, judicial decision support, and policy research.
  arXiv  Detail & Related papers  (2025-08-29T17:38:47Z)
• Can Language Models Discover Scaling Laws? [57.794209392781845]
  This paper introduces SLDAgent, an evolution-based agent that co-optimize the scaling law model and the parameters, enabling it to autonomously explore complex relationships between variables.<n>For the first time, we demonstrate that SLDAgent can automatically discover laws that exhibit consistently more accurate extrapolation than their established, human-derived counterparts.
  arXiv  Detail & Related papers  (2025-07-27T05:45:26Z)
• Augmented Question-guided Retrieval (AQgR) of Indian Case Law with LLM, RAG, and Structured Summaries [0.0]
  This paper proposes the use of Large Language Models (LLMs) to facilitate the retrieval of relevant cases.<n>Our approach combines Retrieval Augmented Generation (RAG) with structured summaries optimized for Indian case law.<n>The system generates targeted legal questions based on factual scenarios to identify relevant case law more effectively.
  arXiv  Detail & Related papers  (2025-07-23T05:24:44Z)
• UQLegalAI@COLIEE2025: Advancing Legal Case Retrieval with Large Language Models and Graph Neural Networks [26.294747463024017]
  Legal case retrieval plays a pivotal role in the legal domain by facilitating the efficient identification of relevant cases.<n>The Competition on Legal Information Extraction and Entailment (COLIEE) is held annually, offering updated benchmark datasets for evaluation.<n>This paper presents a detailed description of CaseLink, the method employed by UQLegalAI, the second highest team in Task 1 of COLIEE 2025.
  arXiv  Detail & Related papers  (2025-05-27T05:32:50Z)
• LegalAgentBench: Evaluating LLM Agents in Legal Domain [53.70993264644004]
  LegalAgentBench is a benchmark specifically designed to evaluate LLM Agents in the Chinese legal domain.<n>LegalAgentBench includes 17 corpora from real-world legal scenarios and provides 37 tools for interacting with external knowledge.
  arXiv  Detail & Related papers  (2024-12-23T04:02:46Z)
• Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study [9.30538764385435]
  Large Language Models (LLMs) have demonstrated strong potential across legal tasks, yet the problem of legal citation prediction remains under-explored.<n>We introduce the AusLaw Citation Benchmark, a real-world dataset comprising 55k Australian legal instances and 18,677 unique citations.<n>We then conduct a systematic benchmarking across a range of solutions.<n>Results show that neither general nor law-specific LLMs suffice as stand-alone solutions, with performance near zero.
  arXiv  Detail & Related papers  (2024-12-09T07:46:14Z)
• OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models [76.59316249991657]
  Large language models (LLMs) for code have become indispensable in various domains, including code generation, reasoning tasks and agent systems.<n>While open-access code LLMs are increasingly approaching the performance levels of proprietary models, high-quality code LLMs remain limited.<n>We introduce OpenCoder, a top-tier code LLM that not only achieves performance comparable to leading models but also serves as an "open cookbook" for the research community.
  arXiv  Detail & Related papers  (2024-11-07T17:47:25Z)
• Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs [67.54302101989542]
  Legal case retrieval aims to provide similar cases as references for a given fact description. Existing works mainly focus on case-to-case retrieval using lengthy queries. Data scale is insufficient to satisfy the training requirements of existing data-hungry neural models.
  arXiv  Detail & Related papers  (2024-10-09T06:26:39Z)
• LawLLM: Law Large Language Model for the US Legal System [43.13850456765944]
  We introduce the Law Large Language Model (LawLLM), a multi-task model specifically designed for the US legal domain. LawLLM excels at Similar Case Retrieval (SCR), Precedent Case Recommendation (PCR), and Legal Judgment Prediction (LJP) We propose customized data preprocessing techniques for each task that transform raw legal data into a trainable format.
  arXiv  Detail & Related papers  (2024-07-27T21:51:30Z)
• Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration [52.57055162778548]
  Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
  arXiv  Detail & Related papers  (2023-10-13T16:47:20Z)
• Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networks [3.5880535198436156]
  We propose a novel graph-augmented dense statute retriever (G-DSR) model that incorporates the structure of legislation via a graph neural network to improve dense retrieval performance. Experimental results show that our approach outperforms strong retrieval baselines on a real-world expert-annotated SAR dataset.
  arXiv  Detail & Related papers  (2023-01-30T12:59:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.