Towards Explainability in Legal Outcome Prediction Models

• URL: http://arxiv.org/abs/2403.16852v2
• Date: Tue, 16 Apr 2024 03:53:28 GMT
• Title: Towards Explainability in Legal Outcome Prediction Models
• Authors: Josef Valvoda, Ryan Cotterell,
• Abstract summary: We argue that precedent is a natural way of facilitating explainability for legal NLP models. By developing a taxonomy of legal precedent, we are able to compare human judges and neural models. We find that while the models learn to predict outcomes reasonably well, their use of precedent is unlike that of human judges.
• Score: 64.00172507827499
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Current legal outcome prediction models - a staple of legal NLP - do not explain their reasoning. However, to employ these models in the real world, human legal actors need to be able to understand the model's decisions. In the case of common law, legal practitioners reason towards the outcome of a case by referring to past case law, known as precedent. We contend that precedent is, therefore, a natural way of facilitating explainability for legal NLP models. In this paper, we contribute a novel method for identifying the precedent employed by legal outcome prediction models. Furthermore, by developing a taxonomy of legal precedent, we are able to compare human judges and neural models with respect to the different types of precedent they rely on. We find that while the models learn to predict outcomes reasonably well, their use of precedent is unlike that of human judges.

Related papers

• PILOT: Legal Case Outcome Prediction with Case Law [43.680862577060765]
  We identify two unique challenges in making legal case outcome predictions with case law. First, it is crucial to identify relevant precedent cases that serve as fundamental evidence for judges during decision-making. Second, it is necessary to consider the evolution of legal principles over time, as early cases may adhere to different legal contexts.
  arXiv  Detail & Related papers  (2024-01-28T21:18:05Z)
• Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration [52.57055162778548]
  Legal Judgment Prediction (LJP) has become an increasingly crucial task in Legal AI. Precedents are the previous legal cases with similar facts, which are the basis for the judgment of the subsequent case in national legal systems. Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
  arXiv  Detail & Related papers  (2023-10-13T16:47:20Z)
• Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction [46.71918729837462]
  Given the fact description text of a legal case, legal judgment prediction aims to predict the case's charge, law article and penalty term. Previous studies fail to distinguish different classification errors with a standard cross-entropy classification loss. We propose a moco-based supervised contrastive learning to learn distinguishable representations. We further enhance the representation of the fact description with extracted crime amounts which are encoded by a pre-trained numeracy model.
  arXiv  Detail & Related papers  (2022-11-15T15:53:56Z)
• Do Charge Prediction Models Learn Legal Theory? [59.74220430434435]
  We argue that trustworthy charge prediction models should take legal theories into consideration. We propose three principles for trustworthy models should follow in this task, which are sensitive, selective, and presumption of innocence. Our findings indicate that, while existing charge prediction models meet the selective principle on a benchmark dataset, most of them are still not sensitive enough and do not satisfy the presumption of innocence.
  arXiv  Detail & Related papers  (2022-10-31T07:32:12Z)
• JUSTICE: A Benchmark Dataset for Supreme Court's Judgment Prediction [0.0]
  We aim to create a high-quality dataset of SCOTUS court cases so that they may be readily used in natural language processing (NLP) research and other data-driven applications. By using advanced NLP algorithms to analyze previous court cases, the trained models are able to predict and classify a court's judgment.
  arXiv  Detail & Related papers  (2021-12-06T23:19:08Z)
• Predicting Indian Supreme Court Judgments, Decisions, Or Appeals [0.403831199243454]
  We introduce our newly developed ML-enabled legal prediction model and its operational prototype, eLegPredict. eLegPredict is trained and tested over 3072 supreme court cases and has achieved 76% accuracy (F1-score) The eLegPredict is equipped with a mechanism to aid end users, where as soon as a document with new case description is dropped into a designated directory, the system quickly reads through its content and generates prediction.
  arXiv  Detail & Related papers  (2021-09-28T18:28:43Z)
• AutoLAW: Augmented Legal Reasoning through Legal Precedent Prediction [0.0]
  This paper demonstrate how NLP can be used to address an unmet need of the legal community and increase access to justice. The paper introduces Legal Precedent Prediction (LPP), the task of predicting relevant passages from precedential court decisions given the context of a legal argument.
  arXiv  Detail & Related papers  (2021-06-30T13:01:33Z)
• What About the Precedent: An Information-Theoretic Analysis of Common Law [64.49276556192073]
  In common law, the outcome of a new case is determined mostly by precedent cases, rather than existing statutes. We are the first to approach this question by comparing two longstanding jurisprudential views. We find that the precedent's arguments share 0.38 nats of information with the case's outcome, whereas precedent's facts only share 0.18 nats of information.
  arXiv  Detail & Related papers  (2021-04-25T11:20:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.