Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation

• URL: http://arxiv.org/abs/2310.11049v1
• Date: Tue, 17 Oct 2023 07:35:11 GMT
• Title: Nonet at SemEval-2023 Task 6: Methodologies for Legal Evaluation
• Authors: Shubham Kumar Nigam, Aniket Deroy, Noel Shallum, Ayush Kumar Mishra, Anup Roy, Shubham Kumar Mishra, Arnab Bhattacharya, Saptarshi Ghosh, and Kripabandhu Ghosh
• Abstract summary: This paper describes our submission to the SemEval-2023 for Task 6 on LegalEval: Understanding Legal Texts. Our submission concentrated on three subtasks: Legal Named Entity Recognition (L-NER) for Task-B, Legal Judgment Prediction (LJP) for Task-C1, and Court Judgment Prediction with Explanation (CJPE) for Task-C2. Our team obtained competitive rankings of 15$th$, 11$th$, and 1$st$ in Task-B, Task-C1, and Task-C2, respectively, as reported
• Score: 6.454861724309361
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This paper describes our submission to the SemEval-2023 for Task 6 on LegalEval: Understanding Legal Texts. Our submission concentrated on three subtasks: Legal Named Entity Recognition (L-NER) for Task-B, Legal Judgment Prediction (LJP) for Task-C1, and Court Judgment Prediction with Explanation (CJPE) for Task-C2. We conducted various experiments on these subtasks and presented the results in detail, including data statistics and methodology. It is worth noting that legal tasks, such as those tackled in this research, have been gaining importance due to the increasing need to automate legal analysis and support. Our team obtained competitive rankings of 15$^{th}$, 11$^{th}$, and 1$^{st}$ in Task-B, Task-C1, and Task-C2, respectively, as reported on the leaderboard.

Related papers

• SemEval-2024 Task 8: Multidomain, Multimodel and Multilingual Machine-Generated Text Detection [68.858931667807]
  Subtask A is a binary classification task determining whether a text is written by a human or generated by a machine. Subtask B is to detect the exact source of a text, discerning whether it is written by a human or generated by a specific LLM. Subtask C aims to identify the changing point within a text, at which the authorship transitions from human to machine.
  arXiv  Detail & Related papers  (2024-04-22T13:56:07Z)
• CAPTAIN at COLIEE 2023: Efficient Methods for Legal Information Retrieval and Entailment Tasks [7.0271825812050555]
  This paper outlines our strategies for tackling Task 2, Task 3, and Task 4 in the COLIEE 2023 competition. Our approach involved utilizing appropriate state-of-the-art deep learning methods, designing methods based on domain characteristics observation, and applying meticulous engineering practices and methodologies to the competition.
  arXiv  Detail & Related papers  (2024-01-07T17:23:27Z)
• SemEval 2023 Task 6: LegalEval - Understanding Legal Texts [2.172613863157655]
  There is a need for developing NLP-based techniques for processing and automatically understanding legal documents. LegalEval task has three sub-tasks: Task-A (Rhetorical Roles Labeling) is about automatically structuring legal documents into semantically coherent units, Task-B (Legal Named Entity Recognition) deals with identifying relevant entities in a legal document, Task-C (Court Judgement Prediction with Explanation) explores the possibility of automatically predicting the outcome of a legal case. In each of the sub-tasks, the proposed systems outperformed the baselines; however, there is a lot of scope for
  arXiv  Detail & Related papers  (2023-04-19T10:28:32Z)
• Understand Legal Documents with Contextualized Large Language Models [16.416510744265086]
  We present our systems for SemEval-2023 Task 6: understanding legal texts. We first develop the Legal-BERT-HSLN model that considers the comprehensive context information in both intra- and inter-sentence levels. We then train a Legal-LUKE model, which is legal-contextualized and entity-aware, to recognize legal entities.
  arXiv  Detail & Related papers  (2023-03-21T18:48:11Z)
• Findings of the WMT 2022 Shared Task on Translation Suggestion [63.457874930232926]
  We report the result of the first edition of the WMT shared task on Translation Suggestion. The task aims to provide alternatives for specific words or phrases given the entire documents generated by machine translation (MT) It consists two sub-tasks, namely, the naive translation suggestion and translation suggestion with hints.
  arXiv  Detail & Related papers  (2022-11-30T03:48:36Z)
• A Dataset on Malicious Paper Bidding in Peer Review [84.68308372858755]
  Malicious reviewers strategically bid in order to unethically manipulate the paper assignment. A critical impediment towards creating and evaluating methods to mitigate this issue is the lack of publicly-available data on malicious paper bidding. We release a novel dataset, collected from a mock conference activity where participants were instructed to bid either honestly or maliciously.
  arXiv  Detail & Related papers  (2022-06-24T20:23:33Z)
• nigam@COLIEE-22: Legal Case Retrieval and Entailment using Cascading of Lexical and Semantic-based models [0.951828574518325]
  This paper describes our submission to the Competition on Legal Information Extraction/Entailment 2022 (COLIEE-2022) workshop on case law competition for tasks 1 and 2. We employed the neural models Sentence-BERT and Sent2Vec for semantic understanding and the traditional retrieval model BM25 for exact matching in both tasks.
  arXiv  Detail & Related papers  (2022-04-16T18:10:02Z)
• A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges [73.34944216896837]
  Legal judgment prediction (LJP) applies Natural Language Processing (NLP) techniques to predict judgment results based on fact descriptions automatically. We analyze 31 LJP datasets in 6 languages, present their construction process and define a classification method of LJP. We show the state-of-art results for 8 representative datasets from different court cases and discuss the open challenges.
  arXiv  Detail & Related papers  (2022-04-11T04:06:28Z)
• IAM: A Comprehensive and Large-Scale Dataset for Integrated Argument Mining Tasks [59.457948080207174]
  In this work, we introduce a comprehensive and large dataset named IAM, which can be applied to a series of argument mining tasks. Near 70k sentences in the dataset are fully annotated based on their argument properties. We propose two new integrated argument mining tasks associated with the debate preparation process: (1) claim extraction with stance classification (CESC) and (2) claim-evidence pair extraction (CEPE)
  arXiv  Detail & Related papers  (2022-03-23T08:07:32Z)
• THUIR@COLIEE-2020: Leveraging Semantic Understanding and Exact Matching for Legal Case Retrieval and Entailment [41.51705651274111]
  We present our methodologies for tackling the challenges of legal case retrieval and entailment. We participated in the two case law tasks, i.e., the legal case retrieval task and the legal case entailment task. In both tasks, we employed the neural models for semantic understanding and the traditional retrieval models for exact matching.
  arXiv  Detail & Related papers  (2020-12-24T04:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.