Automatic Information Extraction From Employment Tribunal Judgements Using Large Language Models

• URL: http://arxiv.org/abs/2403.12936v1
• Date: Tue, 19 Mar 2024 17:43:08 GMT
• Title: Automatic Information Extraction From Employment Tribunal Judgements Using Large Language Models
• Authors: Joana Ribeiro de Faria, Huiyuan Xie, Felix Steffek,
• Abstract summary: This paper presents a study on the application of GPT-4, a large language model, for automatic information extraction from UK employment tribunal cases. We meticulously evaluated GPT-4's performance in extracting critical information with a manual verification process.
• Score: 0.4810407297181484
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Court transcripts and judgments are rich repositories of legal knowledge, detailing the intricacies of cases and the rationale behind judicial decisions. The extraction of key information from these documents provides a concise overview of a case, crucial for both legal experts and the public. With the advent of large language models (LLMs), automatic information extraction has become increasingly feasible and efficient. This paper presents a comprehensive study on the application of GPT-4, a large language model, for automatic information extraction from UK Employment Tribunal (UKET) cases. We meticulously evaluated GPT-4's performance in extracting critical information with a manual verification process to ensure the accuracy and relevance of the extracted data. Our research is structured around two primary extraction tasks: the first involves a general extraction of eight key aspects that hold significance for both legal specialists and the general public, including the facts of the case, the claims made, references to legal statutes, references to precedents, general case outcomes and corresponding labels, detailed order and remedies and reasons for the decision. The second task is more focused, aimed at analysing three of those extracted features, namely facts, claims and outcomes, in order to facilitate the development of a tool capable of predicting the outcome of employment law disputes. Through our analysis, we demonstrate that LLMs like GPT-4 can obtain high accuracy in legal information extraction, highlighting the potential of LLMs in revolutionising the way legal information is processed and utilised, offering significant implications for legal research and practice.

Related papers

• JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking [81.88787401178378]
  We introduce JudgeRank, a novel agentic reranker that emulates human cognitive processes when assessing document relevance. We evaluate JudgeRank on the reasoning-intensive BRIGHT benchmark, demonstrating substantial performance improvements over first-stage retrieval methods. In addition, JudgeRank performs on par with fine-tuned state-of-the-art rerankers on the popular BEIR benchmark, validating its zero-shot generalization capability.
  arXiv  Detail & Related papers  (2024-10-31T18:43:12Z)
• Learning Interpretable Legal Case Retrieval via Knowledge-Guided Case Reformulation [22.85652668826498]
  This paper introduces KELLER, a legal knowledge-guided case reformulation approach based on large language models (LLMs) By incorporating professional legal knowledge about crimes and law articles, we enable large language models to accurately reformulate the original legal case into concise sub-facts of crimes.
  arXiv  Detail & Related papers  (2024-06-28T08:59:45Z)
• InternLM-Law: An Open Source Chinese Legal Large Language Model [72.2589401309848]
  InternLM-Law is a specialized LLM tailored for addressing diverse legal queries related to Chinese laws. We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries. InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks.
  arXiv  Detail & Related papers  (2024-06-21T06:19:03Z)
• DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
  We introduce DELTA, a discriminative model designed for legal case retrieval. We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability. Our approach can outperform existing state-of-the-art methods in legal case retrieval.
  arXiv  Detail & Related papers  (2024-03-27T10:40:14Z)
• Leveraging Large Language Models for Relevance Judgments in Legal Case Retrieval [18.058942674792604]
  We propose a novel few-shot workflow tailored to the relevant judgment of legal cases. By comparing the relevance judgments of LLMs and human experts, we empirically show that we can obtain reliable relevance judgments.
  arXiv  Detail & Related papers  (2024-03-27T09:46:56Z)
• Low-Resource Court Judgment Summarization for Common Law Systems [32.13166048504629]
  We present CLSum, the first dataset for summarizing multi-jurisdictional common law court judgment documents. This is the first court judgment summarization work adopting large language models (LLMs) in data augmentation, summary generation, and evaluation.
  arXiv  Detail & Related papers  (2024-03-07T12:47:42Z)
• LLM vs. Lawyers: Identifying a Subset of Summary Judgments in a Large UK Case Law Dataset [0.0]
  This study addresses the gap in the literature working with large legal corpora about how to isolate cases, in our case summary judgments, from a large corpus of UK court decisions. We use the Cambridge Law Corpus of 356,011 UK court decisions and determine that the large language model achieves a weighted F1 score of 0.94 versus 0.78 for keywords. We identify and extract 3,102 summary judgment cases, enabling us to map their distribution across various UK courts over a temporal span.
  arXiv  Detail & Related papers  (2024-03-04T10:13:30Z)
• Legal Summarisation through LLMs: The PRODIGIT Project [4.840725842638346]
  PRODIGIT aims to support tax judges and lawyers through digital technology, focusing on AI. We have focused on generation of summaries of judicial decisions and on the extraction of related information. We have deployed and evaluated different tools and approaches to extractive and abstractive summarisation.
  arXiv  Detail & Related papers  (2023-08-04T16:59:48Z)
• Leveraging Large Language Models for Topic Classification in the Domain of Public Affairs [65.9077733300329]
  Large Language Models (LLMs) have the potential to greatly enhance the analysis of public affairs documents. LLMs can be of great use to process domain-specific documents, such as those in the domain of public affairs.
  arXiv  Detail & Related papers  (2023-06-05T13:35:01Z)
• SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval [75.05173891207214]
  Legal case retrieval plays a core role in the intelligent legal system. Most existing language models have difficulty understanding the long-distance dependencies between different structures. We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
  arXiv  Detail & Related papers  (2023-04-22T10:47:01Z)
• A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges [73.34944216896837]
  Legal judgment prediction (LJP) applies Natural Language Processing (NLP) techniques to predict judgment results based on fact descriptions automatically. We analyze 31 LJP datasets in 6 languages, present their construction process and define a classification method of LJP. We show the state-of-art results for 8 representative datasets from different court cases and discuss the open challenges.
  arXiv  Detail & Related papers  (2022-04-11T04:06:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.