Low-Resource Court Judgment Summarization for Common Law Systems
- URL: http://arxiv.org/abs/2403.04454v1
- Date: Thu, 7 Mar 2024 12:47:42 GMT
- Title: Low-Resource Court Judgment Summarization for Common Law Systems
- Authors: Shuaiqi Liu, Jiannong Cao, Yicong Li, Ruosong Yang, Zhiyuan Wen
- Abstract summary: We present CLSum, the first dataset for summarizing multi-jurisdictional common law court judgment documents.
This is the first court judgment summarization work adopting large language models (LLMs) in data augmentation, summary generation, and evaluation.
- Score: 32.13166048504629
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Common law courts need to refer to similar precedents' judgments to inform
their current decisions. Generating high-quality summaries of court judgment
documents can facilitate legal practitioners to efficiently review previous
cases and assist the general public in accessing how the courts operate and how
the law is applied. Previous court judgment summarization research focuses on
civil law or a particular jurisdiction's judgments. However, judges can refer
to the judgments from all common law jurisdictions. Current summarization
datasets are insufficient to satisfy the demands of summarizing precedents
across multiple jurisdictions, especially when labeled data are scarce for many
jurisdictions. To address the lack of datasets, we present CLSum, the first
dataset for summarizing multi-jurisdictional common law court judgment
documents. Besides, this is the first court judgment summarization work
adopting large language models (LLMs) in data augmentation, summary generation,
and evaluation. Specifically, we design an LLM-based data augmentation method
incorporating legal knowledge. We also propose a legal knowledge enhanced
evaluation metric based on LLM to assess the quality of generated judgment
summaries. Our experimental results verify that the LLM-based summarization
methods can perform well in the few-shot and zero-shot settings. Our LLM-based
data augmentation method can mitigate the impact of low data resources.
Furthermore, we carry out comprehensive comparative experiments to find
essential model components and settings that are capable of enhancing
summarization performance.
Related papers
- JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking [81.88787401178378]
We introduce JudgeRank, a novel agentic reranker that emulates human cognitive processes when assessing document relevance.
We evaluate JudgeRank on the reasoning-intensive BRIGHT benchmark, demonstrating substantial performance improvements over first-stage retrieval methods.
In addition, JudgeRank performs on par with fine-tuned state-of-the-art rerankers on the popular BEIR benchmark, validating its zero-shot generalization capability.
arXiv Detail & Related papers (2024-10-31T18:43:12Z) - Beyond Borders: Investigating Cross-Jurisdiction Transfer in Legal Case Summarization [2.9612936741643705]
We explore the cross-jurisdictional generalizability of legal case summarization models.
Specifically, we explore how to effectively summarize legal cases of a target jurisdiction where reference summaries are not available.
arXiv Detail & Related papers (2024-03-28T11:18:31Z) - DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
We introduce DELTA, a discriminative model designed for legal case retrieval.
We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability.
Our approach can outperform existing state-of-the-art methods in legal case retrieval.
arXiv Detail & Related papers (2024-03-27T10:40:14Z) - Leveraging Large Language Models for Relevance Judgments in Legal Case Retrieval [18.058942674792604]
We propose a novel few-shot workflow tailored to the relevant judgment of legal cases.
By comparing the relevance judgments of LLMs and human experts, we empirically show that we can obtain reliable relevance judgments.
arXiv Detail & Related papers (2024-03-27T09:46:56Z) - LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK
Case Law Dataset [0.0]
This study addresses the gap in the literature working with large legal corpora about how to isolate cases, in our case summary judgments, from a large corpus of UK court decisions.
We use the Cambridge Law Corpus of 356,011 UK court decisions and determine that the large language model achieves a weighted F1 score of 0.94 versus 0.78 for keywords.
We identify and extract 3,102 summary judgment cases, enabling us to map their distribution across various UK courts over a temporal span.
arXiv Detail & Related papers (2024-03-04T10:13:30Z) - MUSER: A Multi-View Similar Case Retrieval Dataset [65.36779942237357]
Similar case retrieval (SCR) is a representative legal AI application that plays a pivotal role in promoting judicial fairness.
Existing SCR datasets only focus on the fact description section when judging the similarity between cases.
We present M, a similar case retrieval dataset based on multi-view similarity measurement and comprehensive legal element with sentence-level legal element annotations.
arXiv Detail & Related papers (2023-10-24T08:17:11Z) - A Comprehensive Evaluation of Large Language Models on Legal Judgment
Prediction [60.70089334782383]
Large language models (LLMs) have demonstrated great potential for domain-specific applications.
Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks.
We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
arXiv Detail & Related papers (2023-10-18T07:38:04Z) - Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model
Collaboration [52.57055162778548]
Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI.
Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems.
Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
arXiv Detail & Related papers (2023-10-13T16:47:20Z) - JUSTICE: A Benchmark Dataset for Supreme Court's Judgment Prediction [0.0]
We aim to create a high-quality dataset of SCOTUS court cases so that they may be readily used in natural language processing (NLP) research and other data-driven applications.
By using advanced NLP algorithms to analyze previous court cases, the trained models are able to predict and classify a court's judgment.
arXiv Detail & Related papers (2021-12-06T23:19:08Z) - Legal Judgment Prediction with Multi-Stage CaseRepresentation Learning
in the Real Court Setting [25.53133777558123]
We introduce a novel dataset from real courtrooms to predict the legal judgment in a reasonably encyclopedic manner.
An extensive set of experiments with a large civil trial data set shows that the proposed model can more accurately characterize the interactions among claims, fact and debate for legal judgment prediction.
arXiv Detail & Related papers (2021-07-12T04:27:14Z) - Equality before the Law: Legal Judgment Consistency Analysis for
Fairness [55.91612739713396]
In this paper, we propose an evaluation metric for judgment inconsistency, Legal Inconsistency Coefficient (LInCo)
We simulate judges from different groups with legal judgment prediction (LJP) models and measure the judicial inconsistency with the disagreement of the judgment results given by LJP models trained on different groups.
We employ LInCo to explore the inconsistency in real cases and come to the following observations: (1) Both regional and gender inconsistency exist in the legal system, but gender inconsistency is much less than regional inconsistency.
arXiv Detail & Related papers (2021-03-25T14:28:00Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.