ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the US

• URL: http://arxiv.org/abs/2211.00582v1
• Date: Tue, 1 Nov 2022 16:57:59 GMT
• Title: ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the US
• Authors: Gil Semo, Dor Bernsohn, Ben Hagag, Gila Hayat, Joel Niklaus
• Abstract summary: We release for the first time a challenging LJP dataset focused on class action cases in the US. It is the first dataset in the common law system that focuses on the harder and more realistic task involving the complaints as input instead of the often used facts summary written by the court. Our Longformer model clearly outperforms the human baseline (63%), despite only considering the first 2,048 tokens. Furthermore, we perform a detailed error analysis and find that the Longformer model is significantly better calibrated than the human experts.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The research field of Legal Natural Language Processing (NLP) has been very active recently, with Legal Judgment Prediction (LJP) becoming one of the most extensively studied tasks. To date, most publicly released LJP datasets originate from countries with civil law. In this work, we release, for the first time, a challenging LJP dataset focused on class action cases in the US. It is the first dataset in the common law system that focuses on the harder and more realistic task involving the complaints as input instead of the often used facts summary written by the court. Additionally, we study the difficulty of the task by collecting expert human predictions, showing that even human experts can only reach 53% accuracy on this dataset. Our Longformer model clearly outperforms the human baseline (63%), despite only considering the first 2,048 tokens. Furthermore, we perform a detailed error analysis and find that the Longformer model is significantly better calibrated than the human experts. Finally, we publicly release the dataset and the code used for the experiments.

Related papers

• Towards Explainability in Legal Outcome Prediction Models [64.00172507827499]
  We argue that precedent is a natural way of facilitating explainability for legal NLP models. By developing a taxonomy of legal precedent, we are able to compare human judges and neural models. We find that while the models learn to predict outcomes reasonably well, their use of precedent is unlike that of human judges.
  arXiv  Detail & Related papers  (2024-03-25T15:15:41Z)
• Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration [52.57055162778548]
  Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
  arXiv  Detail & Related papers  (2023-10-13T16:47:20Z)
• Adversarially Robust Neural Legal Judgement Systems [0.0]
  Legal judgment prediction is a task of predicting the outcome of court cases on a given text description of facts of cases. For such systems to be practically helpful, they should be robust from adversarial attacks. We propose an approach for making robust Legal Judgement Prediction systems.
  arXiv  Detail & Related papers  (2023-07-31T21:44:48Z)
• Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction [46.71918729837462]
  Given the fact description text of a legal case, legal judgment prediction aims to predict the case's charge, law article and penalty term. Previous studies fail to distinguish different classification errors with a standard cross-entropy classification loss. We propose a moco-based supervised contrastive learning to learn distinguishable representations. We further enhance the representation of the fact description with extracted crime amounts which are encoded by a pre-trained numeracy model.
  arXiv  Detail & Related papers  (2022-11-15T15:53:56Z)
• A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges [73.34944216896837]
  Legal judgment prediction (LJP) applies Natural Language Processing (NLP) techniques to predict judgment results based on fact descriptions automatically. We analyze 31 LJP datasets in 6 languages, present their construction process and define a classification method of LJP. We show the state-of-art results for 8 representative datasets from different court cases and discuss the open challenges.
  arXiv  Detail & Related papers  (2022-04-11T04:06:28Z)
• LEVEN: A Large-Scale Chinese Legal Event Detection Dataset [82.44096140591675]
  We present LEVEN, a large-scale Chinese LEgal eVENt detection dataset, with 8,116 legal documents and 150,977 human-annotated event mentions in 108 event types. LEVEN is the largest Legal Event Detection dataset and has dozens of times the data scale of others, which shall significantly promote the training and evaluation of LED methods.
  arXiv  Detail & Related papers  (2022-03-16T11:40:02Z)
• Sequential Multi-task Learning with Task Dependency for Appeal Judgment Prediction [28.505366852202794]
  Legal Judgment Prediction (LJP) aims to automatically predict judgment results, such as charges, relevant law articles, and the term of penalty. This paper concerns a worthwhile but not well-studied LJP task, Appeal judgment Prediction (AJP), which predicts the judgment of an appellate court on an appeal case. We propose a Sequential Multi-task Learning Framework with Task Dependency for Appeal Judgement Prediction (SMAJudge) to address these challenges.
  arXiv  Detail & Related papers  (2022-03-09T08:51:13Z)
• JUSTICE: A Benchmark Dataset for Supreme Court's Judgment Prediction [0.0]
  We aim to create a high-quality dataset of SCOTUS court cases so that they may be readily used in natural language processing (NLP) research and other data-driven applications. By using advanced NLP algorithms to analyze previous court cases, the trained models are able to predict and classify a court's judgment.
  arXiv  Detail & Related papers  (2021-12-06T23:19:08Z)
• Predicting Indian Supreme Court Judgments, Decisions, Or Appeals [0.403831199243454]
  We introduce our newly developed ML-enabled legal prediction model and its operational prototype, eLegPredict. eLegPredict is trained and tested over 3072 supreme court cases and has achieved 76% accuracy (F1-score) The eLegPredict is equipped with a mechanism to aid end users, where as soon as a document with new case description is dropped into a designated directory, the system quickly reads through its content and generates prediction.
  arXiv  Detail & Related papers  (2021-09-28T18:28:43Z)
• Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents [56.40163943394202]
  We release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding. We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering.
  arXiv  Detail & Related papers  (2021-05-09T09:39:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.