LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Text

• URL: http://arxiv.org/abs/2410.12064v1
• Date: Tue, 15 Oct 2024 21:02:44 GMT
• Title: LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Text
• Authors: Ben Hagag, Liav Harpaz, Gil Semo, Dor Bernsohn, Rohit Saha, Pashootan Vaezipoor, Kyryl Truskovskyi, Gerasimos Spanakis,
• Abstract summary: This paper focuses on detecting legal violations within text in the wild across two sub-tasks: LegalLens-NER for identifying legal violation entities and LegalLens-NLI for associating these violations with relevant legal contexts and affected individuals. The top-performing team achieved a 7.11% improvement in NER over the baseline, while NLI saw a more marginal improvement of 5.7%.
• Score: 7.839638824275218
• License:
• Abstract: This paper presents the results of the LegalLens Shared Task, focusing on detecting legal violations within text in the wild across two sub-tasks: LegalLens-NER for identifying legal violation entities and LegalLens-NLI for associating these violations with relevant legal contexts and affected individuals. Using an enhanced LegalLens dataset covering labor, privacy, and consumer protection domains, 38 teams participated in the task. Our analysis reveals that while a mix of approaches was used, the top-performing teams in both tasks consistently relied on fine-tuning pre-trained language models, outperforming legal-specific models and few-shot methods. The top-performing team achieved a 7.11% improvement in NER over the baseline, while NLI saw a more marginal improvement of 5.7%. Despite these gains, the complexity of legal texts leaves room for further advancements.

Related papers

• Bonafide at LegalLens 2024 Shared Task: Using Lightweight DeBERTa Based Encoder For Legal Violation Detection and Resolution [1.2283121128307906]
  We present two systems -- Named Entity Resolution (NER) and Natural Language Inference (NLI) -- for detecting legal violations within unstructured textual data. Both systems are lightweight DeBERTa based encoders that outperform the LLM baselines.
  arXiv  Detail & Related papers  (2024-10-30T12:42:38Z)
• Hybrid Deep Learning for Legal Text Analysis: Predicting Punishment Durations in Indonesian Court Rulings [0.0]
  This study develops a deep learning-based predictive system for court sentence lengths. Our model, combining CNN and BiLSTM with attention mechanism, achieved an R-squared score of 0.5893.
  arXiv  Detail & Related papers  (2024-10-26T07:07:48Z)
• Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence [0.0]
  This paper presents our system description and error analysis of our entry for NLLP 2024 shared task on Legal Natural Language Inference (L-NLI) The task required classifying relationships as entailed, contradicted, or neutral, indicating any association between the review and the complaint. Our system emerged as the winning submission, significantly outperforming other entries with a substantial margin and demonstrating the effectiveness of our approach in legal text analysis.
  arXiv  Detail & Related papers  (2024-10-21T13:20:15Z)
• InternLM-Law: An Open Source Chinese Legal Large Language Model [72.2589401309848]
  InternLM-Law is a specialized LLM tailored for addressing diverse legal queries related to Chinese laws. We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries. InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks.
  arXiv  Detail & Related papers  (2024-06-21T06:19:03Z)
• FLawN-T5: An Empirical Examination of Effective Instruction-Tuning Data Mixtures for Legal Reasoning [47.001169623840354]
  LawInstruct is a large legal instruction dataset covering 17 jurisdictions, 24 languages and a total of 12M examples. We present evidence that domain-specific pretraining and instruction tuning improve performance on LegalBench. LawInstruct is a resource for accelerating the development of models with stronger information processing and decision making capabilities in the legal domain.
  arXiv  Detail & Related papers  (2024-04-02T17:33:34Z)
• DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
  We introduce DELTA, a discriminative model designed for legal case retrieval. We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability. Our approach can outperform existing state-of-the-art methods in legal case retrieval.
  arXiv  Detail & Related papers  (2024-03-27T10:40:14Z)
• LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models [15.98468948605927]
  LegalBench is a benchmark consisting of 162 tasks covering six different types of legal reasoning. This paper describes LegalBench, presents an empirical evaluation of 20 open-source and commercial LLMs, and illustrates the types of research explorations LegalBench enables.
  arXiv  Detail & Related papers  (2023-08-20T22:08:03Z)
• SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval [75.05173891207214]
  Legal case retrieval plays a core role in the intelligent legal system. Most existing language models have difficulty understanding the long-distance dependencies between different structures. We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
  arXiv  Detail & Related papers  (2023-04-22T10:47:01Z)
• SemEval 2023 Task 6: LegalEval - Understanding Legal Texts [2.172613863157655]
  There is a need for developing NLP-based techniques for processing and automatically understanding legal documents. LegalEval task has three sub-tasks: Task-A (Rhetorical Roles Labeling) is about automatically structuring legal documents into semantically coherent units, Task-B (Legal Named Entity Recognition) deals with identifying relevant entities in a legal document, Task-C (Court Judgement Prediction with Explanation) explores the possibility of automatically predicting the outcome of a legal case. In each of the sub-tasks, the proposed systems outperformed the baselines; however, there is a lot of scope for
  arXiv  Detail & Related papers  (2023-04-19T10:28:32Z)
• Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents [56.40163943394202]
  We release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding. We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering.
  arXiv  Detail & Related papers  (2021-05-09T09:39:25Z)
• How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence [81.04070052740596]
  Legal Artificial Intelligence (LegalAI) focuses on applying the technology of artificial intelligence, especially natural language processing, to benefit tasks in the legal domain. This paper introduces the history, the current state, and the future directions of research in LegalAI.
  arXiv  Detail & Related papers  (2020-04-25T14:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.