By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode

• URL: http://arxiv.org/abs/2605.25186v1
• Date: Sun, 24 May 2026 17:31:31 GMT
• Title: By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode
• Abstract summary: We present a method for comparing different formalizations of the same legal provision by their inferences on individual cases.<n>Given multiple formalizations of a provision, we match them at the node level, derive a shared interface for each pair from the matching, and use a SAT solver to enumerate the edge cases on which any two formalizations disagree.<n> Selected edge cases are then verbalized into concrete factual scenarios that a legal expert can examine and act on.
• Score: 9.91190832627458
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Formalizing legal provisions promises machine-accessible law and automated legal reasoning, and recent LLMs make it tempting to generate such formalizations directly from statutory text. However, any formalization makes implicit interpretive choices whose consequences are hard to anticipate, especially if an LLM is the author. We present a method for systematically comparing different formalizations of the same legal provision by their inferences on individual cases. Given multiple formalizations of a provision, we match them at the node level, derive a shared interface for each pair from the matching, and use a SAT solver to enumerate the edge cases on which any two formalizations disagree. Selected edge cases are then verbalized into concrete factual scenarios that a legal expert can examine and act on. We apply our method to formalizations of ten EU provisions generated by nine frontier LLMs. We find that behavioral divergence between formalizations is essentially uncorrelated with their structural agreement and that the verbalized cases reveal qualitatively distinct types of disagreement, including divergences that mirror genuine controversies in the legal commentary.

Related papers

• Closing the Loop: Formally Verified Law as a Reward Signal for Self-Improving Legal AI [0.0]
  This article develops an architecture that creates a formally verifiable reward signal to train legal AI.<n>For the computational components of law, the architecture provides provable correctness.<n>We show that the same architecture has a structural advantage for legal AI training.
  arXiv  Detail & Related papers  (2026-06-22T20:21:49Z)
• Reasoners or Translators? Contamination-aware Evaluation and Neuro-Symbolic Robustness in Tax Law [19.724405436101495]
  We study tax law reasoning approaches and implement a contamination detection protocol to rigorously assess reliability.<n>We show that performance can be inflated by contamination.<n>Our findings indicate that legal reasoning is inherently compositional and that neuro-symbolic frameworks offer a more reliable and robust foundation for legal AI.
  arXiv  Detail & Related papers  (2026-05-15T15:20:16Z)
• Legal2LogicICL: Improving Generalization in Transforming Legal Cases to Logical Formulas via Diverse Few-Shot Learning [1.315248337031741]
  This work aims to improve the generalization of logic-based legal reasoning systems by integrating recent advances in NLP with legal-domain adaptive few-shot learning techniques using LLMs.<n>We introduce Legal2LogicICL, a few-shot retrieval framework that balances diversity and similarity of exemplars at both the latent semantic representation level and the legal text structure level.<n>Our approach significantly improves accuracy, stability, and generalization in transforming natural-language legal case descriptions into logical representations.
  arXiv  Detail & Related papers  (2026-04-13T16:36:48Z)
• LegalOne: A Family of Foundation Models for Reliable Legal Reasoning [54.57434222018289]
  We present LegalOne, a family of foundational models specifically tailored for the Chinese legal domain.<n>LegalOne is developed through a comprehensive three-phase pipeline designed to master legal reasoning.<n>We publicly release the LegalOne weights and the LegalKit evaluation framework to advance the field of Legal AI.
  arXiv  Detail & Related papers  (2026-01-31T10:18:32Z)
• CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval [5.305110876082343]
  CaseFacts is a benchmark for verifying legal claims against U.S. Supreme Court precedents.<n>The dataset consists of 6,294 claims categorized as Supported, Refuted, or Overruled.
  arXiv  Detail & Related papers  (2026-01-23T23:41:46Z)
• Towards Trustworthy Legal AI through LLM Agents and Formal Reasoning [11.842866992683158]
  Existing LLM-based systems excel at surface-level text analysis but lack the guarantees required for principled rationality.<n>We introduce L4M, a novel framework that combines LLM agents with SMT-solver-backed jurisprudence.<n>We show that our system surpasses advanced LLMs including GPT-o4-mini, DeepSeek-V3, and Claude 4 as well as state-of-the-art Legal AI proofs.
  arXiv  Detail & Related papers  (2025-11-26T04:05:06Z)
• RLJP: Legal Judgment Prediction via First-Order Logic Rule-enhanced with Large Language Models [58.69183479148083]
  Legal Judgment Prediction (LJP) is a pivotal task in legal AI.<n>Existing LJP models integrate judicial precedents and legal knowledge for high performance.<n>But they neglect legal reasoning logic, a critical component of legal judgments requiring rigorous logical analysis.<n>This paper proposes a rule-enhanced legal judgment prediction framework based on first-order logic (FOL) formalism and comparative learning (CL)
  arXiv  Detail & Related papers  (2025-05-27T14:50:21Z)
• Legal Rule Induction: Towards Generalizable Principle Discovery from Analogous Judicial Precedents [39.35255423087048]
  Legal rules encompass not only codified statutes but also implicit adjudicatory principles derived from precedents that contain discretionary norms, social morality, and policy.<n>We formalize Legal Rule Induction (LRI) as the task of deriving concise, generalizable doctrinal rules from sets of analogous precedents.<n>We introduce the first LRI benchmark, comprising 5,121 case sets (38,088 Chinese cases in total) for model tuning and 216 expert-annotated gold test sets.
  arXiv  Detail & Related papers  (2025-05-20T09:10:52Z)
• Evaluating the Correctness of Inference Patterns Used by LLMs for Judgment [53.17596274334017]
  We evaluate the correctness of the detailed inference patterns of an LLM behind its seemingly correct outputs.<n>Experiments show that even when the language generation results appear correct, a significant portion of the inference patterns used by the LLM for the legal judgment may represent misleading or irrelevant logic.
  arXiv  Detail & Related papers  (2024-10-06T08:33:39Z)
• DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
  We introduce DELTA, a discriminative model designed for legal case retrieval. We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability. Our approach can outperform existing state-of-the-art methods in legal case retrieval.
  arXiv  Detail & Related papers  (2024-03-27T10:40:14Z)
• Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration [52.57055162778548]
  Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
  arXiv  Detail & Related papers  (2023-10-13T16:47:20Z)
• SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval [75.05173891207214]
  Legal case retrieval plays a core role in the intelligent legal system. Most existing language models have difficulty understanding the long-distance dependencies between different structures. We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
  arXiv  Detail & Related papers  (2023-04-22T10:47:01Z)
• Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction [46.71918729837462]
  Given the fact description text of a legal case, legal judgment prediction aims to predict the case's charge, law article and penalty term. Previous studies fail to distinguish different classification errors with a standard cross-entropy classification loss. We propose a moco-based supervised contrastive learning to learn distinguishable representations. We further enhance the representation of the fact description with extracted crime amounts which are encoded by a pre-trained numeracy model.
  arXiv  Detail & Related papers  (2022-11-15T15:53:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.