CrossLex: A Source-Grounded Benchmark for Cross-Jurisdictional Legal Reasoning in Large Language Models

• URL: http://arxiv.org/abs/2608.01292v1
• Date: Sun, 02 Aug 2026 15:03:09 GMT
• Title: CrossLex: A Source-Grounded Benchmark for Cross-Jurisdictional Legal Reasoning in Large Language Models
• Abstract summary: We introduce CrossLex, a same-fact, legal-source-grounded benchmark for evaluating cross-jurisdictional legal reasoning.<n>Built from authoritative legal sources, CrossLex aligns 55 legal issues spanning contract, consumer, criminal, family, and labor law, and constructs jurisdiction-aligned questions.<n>To disentangle basic legal knowledge from cross-jurisdictional reasoning, CrossLex defines three complementary tasks: single-jurisdiction reasoning (T1), joint cross-jurisdictional comparison (T2), and fine-grained cross-jurisdictional evaluation (T3)
• Score: 8.433017542591221
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Legal reasoning is inherently jurisdiction-dependent: the same facts can call for different legal rules and yield different conclusions across legal systems. Yet existing benchmarks rarely evaluate whether large language models (LLMs) can recognize such jurisdiction-specific variation, especially when identical fact patterns lead to divergent legal outcomes.We introduce CrossLex, a same-fact, legal-source-grounded benchmark for evaluating cross-jurisdictional legal reasoning in LLMs across three jurisdictions: China, California, and Germany. Built from authoritative legal sources, CrossLex aligns 55 legal issues spanning contract, consumer, criminal, family, and labor law, and constructs jurisdiction-aligned questions paired with answers and supporting citations. In total, CrossLex contains 6,149 instances organized into 385 fact groups, with all legal issues, answers, and cited authorities reviewed by legal professionals.To disentangle basic legal knowledge from cross-jurisdictional reasoning, CrossLex defines three complementary tasks: single-jurisdiction reasoning (T1), joint cross-jurisdictional comparison (T2), and fine-grained cross-jurisdictional evaluation (T3). We further propose Grounded Joint, a metric that jointly assesses answer correctness and legal-source grounding, and provide a unified evaluation for streamlined benchmarking. Extensive experiments on representative LLMs show that, although current models can often answer legal questions correctly, they struggle to provide accurate cross-jurisdictional legal citations.We hope that CrossLex will facilitate future research on source-grounded cross-jurisdictional legal reasoning.

Related papers

• LexIssue: Benchmarking Legal Issue Identification in Chinese Civil Litigation [6.798861249866644]
  We introduce a legally grounded hierarchical schema that represents legal issues through both free-form issue descriptions and structured legal categories.<n>We construct LexIssue, a benchmark containing 430 real-world Chinese civil litigation cases and 1,303 expert-annotated disputed legal issues.<n> Experimental results show that retrieval-augmented generation using the constructed legal issue knowledge base consistently improves performance in identifying disputed legal issues.
  arXiv  Detail & Related papers  (2026-09-02T01:49:59Z)
• TW-LegalBench: Measuring Taiwanese Legal Understanding [1.8953545128709688]
  Large language models (LLMs) have shown impressive capabilities across diverse tasks, yet their performance on jurisdiction-specific legal reasoning remains underexplored.<n>We present TW-LegalBench that utilizes Taiwanese legal system's rich official corpus open to the public to fill the gap in evaluating LLMs on Taiwanese law.
  arXiv  Detail & Related papers  (2026-06-17T05:25:21Z)
• WisdomInterrogatory (LuWen): An Open-Source Legal Large Language Model Technical Report [55.27414605169639]
  We present WisdomInterrogatory (LuWen), an open-source Chinese legal language model built upon the Baichuan foundation model.<n>We evaluate LuWen on five representative legal tasks spanning both prediction and generation settings.
  arXiv  Detail & Related papers  (2026-04-08T06:59:07Z)
• Where Experts Disagree, Models Fail: Detecting Implicit Legal Citations in French Court Decisions [3.8449738927037207]
  We focus on implicit citation of the French Civil Code in first-instance court decisions.<n>We show that expert disagreement predicts model failures.<n>Despite these limits, reframing the task as top-k ranking and leveraging multi-model consensus yields 76% precision at k = 200 in an unsupervised setting.
  arXiv  Detail & Related papers  (2026-03-24T09:10:57Z)
• LegalOne: A Family of Foundation Models for Reliable Legal Reasoning [54.57434222018289]
  We present LegalOne, a family of foundational models specifically tailored for the Chinese legal domain.<n>LegalOne is developed through a comprehensive three-phase pipeline designed to master legal reasoning.<n>We publicly release the LegalOne weights and the LegalKit evaluation framework to advance the field of Legal AI.
  arXiv  Detail & Related papers  (2026-01-31T10:18:32Z)
• PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice [67.71760070255425]
  We introduce PLawBench, a practical benchmark for evaluating large language models (LLMs) in legal practice scenarios.<n>PLawBench comprises 850 questions across 13 practical legal scenarios, with each question accompanied by expert-designed evaluation rubrics.<n>Using an LLM-based evaluator aligned with human expert judgments, we evaluate 10 state-of-the-art LLMs.
  arXiv  Detail & Related papers  (2026-01-23T11:36:10Z)
• Evaluating Legal Reasoning Traces with Legal Issue Tree Rubrics [49.3262123849242]
  We introduce LEGIT (LEGal Issue Trees), a novel large-scale (24K instances) expert-level legal reasoning dataset.<n>We convert court judgments into hierarchical trees of opposing parties' arguments and the court's conclusions, which serve as rubrics for evaluating the issue coverage and correctness of the reasoning traces.
  arXiv  Detail & Related papers  (2025-11-30T18:32:43Z)
• Not ready for the bench: LLM legal interpretation is unstable and out of step with human judgments [2.8622281002418357]
  Recent scholarship has proposed that legal practitioners add large language models (LLMs) to their interpretive toolkit.<n>This work offers an empirical argument against LLM interpretation as recently practiced by legal scholars and federal judges.<n>Our investigation in English shows that models do not provide stable interpretive judgments.
  arXiv  Detail & Related papers  (2025-10-29T10:21:25Z)
• CLaw: Benchmarking Chinese Legal Knowledge in Large Language Models - A Fine-grained Corpus and Reasoning Analysis [13.067377421250557]
  Large Language Models (LLMs) are increasingly tasked with analyzing legal texts and citing relevant statutes.<n>This paper introduces CLaw, a novel benchmark specifically engineered to meticulously evaluate LLMs on Chinese legal knowledge and its application in reasoning.
  arXiv  Detail & Related papers  (2025-09-25T14:19:51Z)
• LEXam: Benchmarking Legal Reasoning on 340 Law Exams [76.3521146499006]
  We introduce textscLEXam, a novel benchmark derived from 340 law exams spanning 116 law school courses across a range of subjects and degree levels.<n>The dataset comprises 4,886 law exam questions in English and German, including 2,841 long-form, open-ended questions and 2,045 multiple-choice questions.<n>Our results underscore the effectiveness of the dataset in differentiating between models with varying capabilities.
  arXiv  Detail & Related papers  (2025-05-19T08:48:12Z)
• DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
  We introduce DELTA, a discriminative model designed for legal case retrieval. We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability. Our approach can outperform existing state-of-the-art methods in legal case retrieval.
  arXiv  Detail & Related papers  (2024-03-27T10:40:14Z)
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [15.98468948605927]
  LegalBench is a benchmark consisting of 162 tasks covering six different types of legal reasoning. This paper describes LegalBench, presents an empirical evaluation of 20 open-source and commercial LLMs, and illustrates the types of research explorations LegalBench enables.
  arXiv  Detail & Related papers  (2023-08-20T22:08:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.