Generating Legal Commentaries from Case Databases via Retrieval, Clustering, and Generation

• URL: http://arxiv.org/abs/2605.24534v1
• Date: Sat, 23 May 2026 12:07:29 GMT
• Title: Generating Legal Commentaries from Case Databases via Retrieval, Clustering, and Generation
• Abstract summary: We present a fully automated pipeline that transforms large collections of court decisions into legal commentaries for statutes.<n>We extract paragraph-level chunks, summarize their reasoning, and derive keywords, which are embedded and clustered.<n>Our results show that commentary-like argument mining from court decisions to generate reports can be refreshed within minutes at minimal cost.
• Score: 8.81921009098836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a fully automated pipeline that transforms large collections of court decisions into legal commentaries for statutes - without providing any handcrafted doctrinal framework. Using 4.555 decisions of the German Federal Court of Justice that cite sections 242, 280, 812 and 823 of the German Civil Code (BGB), we extract paragraph-level chunks, summarize their reasoning, and derive keywords, which are embedded and clustered. For each cluster, an LLM generates headings and synthesizes citation-rich sections, which are then merged into coherent commentaries by four state-of-the-art LLMs. We evaluate along five dimensions - topical relevance, heading-match, citation faithfulness, cluster distinction and logical ordering - using both a human expert and an LLM-judge. Our results show that commentary-like argument mining from court decisions to generate reports that can be refreshed within minutes at minimal cost is feasible, yet they highlight limitations arising from restricted sources and the normativity of legal reasoning.

Related papers

• Magis-Bench: Evaluating LLMs on Magistrate-Level Legal Tasks [8.678622777553263]
  Magis-Bench is a benchmark for evaluating magistrate-level writing tasks.<n>It comprises 74 questions from eight examinations conducted between 2023 and 2025.<n>We evaluate 23 state-of-the-art LLMs using an LLM-as-a-judge methodology.
  arXiv  Detail & Related papers  (2026-05-08T20:00:14Z)
• Blind to the Human Touch: Overlap Bias in LLM-Based Summary Evaluation [89.52571224447111]
  Large language model (LLM) judges have often been used alongside traditional, algorithm-based metrics for tasks like summarization.<n>We provide an LLM judge bias analysis as a function of overlap with human-written responses in the domain of summarization.
  arXiv  Detail & Related papers  (2026-02-07T19:39:28Z)
• Copyright Detective: A Forensic System to Evidence LLMs Flickering Copyright Leakage Risks [123.36265437655187]
  Copyright Detective is an interactive forensic system for detecting, analyzing, and visualizing potential copyright risks in LLM outputs.<n>It integrates multiple detection paradigms, including content recall testing, paraphrase-level similarity analysis, persuasive probing, and unlearning verification.
  arXiv  Detail & Related papers  (2026-02-05T03:09:52Z)
• Towards Trustworthy Legal AI through LLM Agents and Formal Reasoning [11.842866992683158]
  Existing LLM-based systems excel at surface-level text analysis but lack the guarantees required for principled rationality.<n>We introduce L4M, a novel framework that combines LLM agents with SMT-solver-backed jurisprudence.<n>We show that our system surpasses advanced LLMs including GPT-o4-mini, DeepSeek-V3, and Claude 4 as well as state-of-the-art Legal AI proofs.
  arXiv  Detail & Related papers  (2025-11-26T04:05:06Z)
• LLM-as-a-Judge is Bad, Based on AI Attempting the Exam Qualifying for the Member of the Polish National Board of Appeal [34.008574054602356]
  The paper describes the structure of the exam, which includes a knowledge test on public procurement law and a written judgment.<n>Several LLMs were tested in closed-book and various Retrieval-Augmented Generation settings.<n>The results show that although the models achieved satisfactory scores in the knowledge test, none met the passing threshold in the practical written part.
  arXiv  Detail & Related papers  (2025-11-06T09:11:20Z)
• CitaLaw: Enhancing LLM with Citations in Legal Domain [5.249003454314636]
  We propose CitaLaw, the first benchmark designed to evaluate LLMs' ability to produce legally sound responses with appropriate citations.<n>CitaLaw features a diverse set of legal questions for both laypersons and practitioners, paired with a comprehensive corpus of law articles and precedent cases as a reference pool.
  arXiv  Detail & Related papers  (2024-12-19T06:14:20Z)
• Legal Minds, Algorithmic Decisions: How LLMs Apply Constitutional Principles in Complex Scenarios [0.0]
  GPT-4 consistently aligns more closely with progressive interpretations of the Constitution. Our experiments reveal a distinct tendency of GPT-4 to favor progressive legal interpretations.
  arXiv  Detail & Related papers  (2024-07-29T07:51:43Z)
• SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal [64.9938658716425]
  SORRY-Bench is a proposed benchmark for evaluating large language models' (LLMs) ability to recognize and reject unsafe user requests.<n>First, existing methods often use coarse-grained taxonomy of unsafe topics, and are over-representing some fine-grained topics.<n>Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations.
  arXiv  Detail & Related papers  (2024-06-20T17:56:07Z)
• Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs [87.34281749422756]
  Large language models (LLMs) have achieved impressive human-like performance across various reasoning tasks. However, their mastery of underlying inferential rules still falls short of human capabilities. We propose a logic scaffolding inferential rule generation framework, to construct an inferential rule base, ULogic.
  arXiv  Detail & Related papers  (2024-02-18T03:38:51Z)
• Sentiment Analysis through LLM Negotiations [58.67939611291001]
  A standard paradigm for sentiment analysis is to rely on a singular LLM and makes the decision in a single round. This paper introduces a multi-LLM negotiation framework for sentiment analysis.
  arXiv  Detail & Related papers  (2023-11-03T12:35:29Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.