SemEval 2023 Task 6: LegalEval - Understanding Legal Texts
- URL: http://arxiv.org/abs/2304.09548v3
- Date: Mon, 1 May 2023 11:33:08 GMT
- Title: SemEval 2023 Task 6: LegalEval - Understanding Legal Texts
- Authors: Ashutosh Modi and Prathamesh Kalamkar and Saurabh Karn and Aman Tiwari
and Abhinav Joshi and Sai Kiran Tanikella and Shouvik Kumar Guha and Sachin
Malhan and Vivek Raghavan
- Abstract summary: There is a need for developing NLP-based techniques for processing and automatically understanding legal documents.
LegalEval task has three sub-tasks: Task-A (Rhetorical Roles Labeling) is about automatically structuring legal documents into semantically coherent units, Task-B (Legal Named Entity Recognition) deals with identifying relevant entities in a legal document, Task-C (Court Judgement Prediction with Explanation) explores the possibility of automatically predicting the outcome of a legal case.
In each of the sub-tasks, the proposed systems outperformed the baselines; however, there is a lot of scope for
- Score: 2.172613863157655
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: In populous countries, pending legal cases have been growing exponentially.
There is a need for developing NLP-based techniques for processing and
automatically understanding legal documents. To promote research in the area of
Legal NLP we organized the shared task LegalEval - Understanding Legal Texts at
SemEval 2023. LegalEval task has three sub-tasks: Task-A (Rhetorical Roles
Labeling) is about automatically structuring legal documents into semantically
coherent units, Task-B (Legal Named Entity Recognition) deals with identifying
relevant entities in a legal document and Task-C (Court Judgement Prediction
with Explanation) explores the possibility of automatically predicting the
outcome of a legal case along with providing an explanation for the prediction.
In total 26 teams (approx. 100 participants spread across the world) submitted
systems paper. In each of the sub-tasks, the proposed systems outperformed the
baselines; however, there is a lot of scope for improvement. This paper
describes the tasks, and analyzes techniques proposed by various teams.
Related papers
- LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Text [7.839638824275218]
This paper focuses on detecting legal violations within text in the wild across two sub-tasks: LegalLens-NER for identifying legal violation entities and LegalLens-NLI for associating these violations with relevant legal contexts and affected individuals.
The top-performing team achieved a 7.11% improvement in NER over the baseline, while NLI saw a more marginal improvement of 5.7%.
arXiv Detail & Related papers (2024-10-15T21:02:44Z) - RegNLP in Action: Facilitating Compliance Through Automated Information Retrieval and Answer Generation [51.998738311700095]
Regulatory documents, characterized by their length, complexity and frequent updates, are challenging to interpret.
RegNLP is a multidisciplinary subfield aimed at simplifying access to and interpretation of regulatory rules and obligations.
ObliQA dataset contains 27,869 questions derived from the Abu Dhabi Global Markets (ADGM) financial regulation document collection.
arXiv Detail & Related papers (2024-09-09T14:44:19Z) - InternLM-Law: An Open Source Chinese Legal Large Language Model [72.2589401309848]
InternLM-Law is a specialized LLM tailored for addressing diverse legal queries related to Chinese laws.
We meticulously construct a dataset in the Chinese legal domain, encompassing over 1 million queries.
InternLM-Law achieves the highest average performance on LawBench, outperforming state-of-the-art models, including GPT-4, on 13 out of 20 subtasks.
arXiv Detail & Related papers (2024-06-21T06:19:03Z) - DELTA: Pre-train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment [55.91429725404988]
We introduce DELTA, a discriminative model designed for legal case retrieval.
We leverage shallow decoders to create information bottlenecks, aiming to enhance the representation ability.
Our approach can outperform existing state-of-the-art methods in legal case retrieval.
arXiv Detail & Related papers (2024-03-27T10:40:14Z) - Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation [6.454861724309361]
This paper describes our submission to the SemEval-2023 for Task 6 on LegalEval: Understanding Legal Texts.
Our submission concentrated on three subtasks: Legal Named Entity Recognition (L-NER) for Task-B, Legal Judgment Prediction (LJP) for Task-C1, and Court Judgment Prediction with Explanation (CJPE) for Task-C2.
Our team obtained competitive rankings of 15$th$, 11$th$, and 1$st$ in Task-B, Task-C1, and Task-C2, respectively, as reported
arXiv Detail & Related papers (2023-10-17T07:35:11Z) - NeCo@ALQAC 2023: Legal Domain Knowledge Acquisition for Low-Resource
Languages through Data Enrichment [2.441072488254427]
This paper presents NeCo Team's solutions to the Vietnamese text processing tasks provided in the Automated Legal Question Answering Competition 2023 (ALQAC 2023)
Our methods for the legal document retrieval task employ a combination of similarity ranking and deep learning models, while for the second task, we propose a range of adaptive techniques to handle different question types.
Our approaches achieve outstanding results on both tasks of the competition, demonstrating the potential benefits and effectiveness of question answering systems in the legal field.
arXiv Detail & Related papers (2023-09-11T14:43:45Z) - SAILER: Structure-aware Pre-trained Language Model for Legal Case
Retrieval [75.05173891207214]
Legal case retrieval plays a core role in the intelligent legal system.
Most existing language models have difficulty understanding the long-distance dependencies between different structures.
We propose a new Structure-Aware pre-traIned language model for LEgal case Retrieval.
arXiv Detail & Related papers (2023-04-22T10:47:01Z) - Understand Legal Documents with Contextualized Large Language Models [16.416510744265086]
We present our systems for SemEval-2023 Task 6: understanding legal texts.
We first develop the Legal-BERT-HSLN model that considers the comprehensive context information in both intra- and inter-sentence levels.
We then train a Legal-LUKE model, which is legal-contextualized and entity-aware, to recognize legal entities.
arXiv Detail & Related papers (2023-03-21T18:48:11Z) - Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents [56.40163943394202]
We release the Longformer-based pre-trained language model, named as Lawformer, for Chinese legal long documents understanding.
We evaluate Lawformer on a variety of LegalAI tasks, including judgment prediction, similar case retrieval, legal reading comprehension, and legal question answering.
arXiv Detail & Related papers (2021-05-09T09:39:25Z) - THUIR@COLIEE-2020: Leveraging Semantic Understanding and Exact Matching
for Legal Case Retrieval and Entailment [41.51705651274111]
We present our methodologies for tackling the challenges of legal case retrieval and entailment.
We participated in the two case law tasks, i.e., the legal case retrieval task and the legal case entailment task.
In both tasks, we employed the neural models for semantic understanding and the traditional retrieval models for exact matching.
arXiv Detail & Related papers (2020-12-24T04:59:45Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.