Bonafide at LegalLens 2024 Shared Task: Using Lightweight DeBERTa Based Encoder For Legal Violation Detection and Resolution

• URL: http://arxiv.org/abs/2410.22977v1
• Date: Wed, 30 Oct 2024 12:42:38 GMT
• Title: Bonafide at LegalLens 2024 Shared Task: Using Lightweight DeBERTa Based Encoder For Legal Violation Detection and Resolution
• Authors: Shikha Bordia,
• Abstract summary: We present two systems -- Named Entity Resolution (NER) and Natural Language Inference (NLI) -- for detecting legal violations within unstructured textual data. Both systems are lightweight DeBERTa based encoders that outperform the LLM baselines.
• Score: 1.2283121128307906
• License:
• Abstract: In this work, we present two systems -- Named Entity Resolution (NER) and Natural Language Inference (NLI) -- for detecting legal violations within unstructured textual data and for associating these violations with potentially affected individuals, respectively. Both these systems are lightweight DeBERTa based encoders that outperform the LLM baselines. The proposed NER system achieved an F1 score of 60.01\% on Subtask A of the LegalLens challenge, which focuses on identifying violations. The proposed NLI system achieved an F1 score of 84.73\% on Subtask B of the LegalLens challenge, which focuses on resolving these violations by matching them with pre-existing legal complaints of class action cases. Our NER system ranked sixth and NLI system ranked fifth on the LegalLens leaderboard. We release the trained models and inference scripts.

Related papers

• uOttawa at LegalLens-2024: Transformer-based Classification Experiments [7.710171464368831]
  This paper presents the methods used for LegalLens-2024 shared task. It focused on detecting legal violations within unstructured textual data and associating these violations with potentially affected individuals. Our results were 86.3% in the L-NER subtask and 88.25% in the L-NLI subtask.
  arXiv  Detail & Related papers  (2024-10-28T15:42:45Z)
• Augmenting Legal Decision Support Systems with LLM-based NLI for Analyzing Social Media Evidence [0.0]
  This paper presents our system description and error analysis of our entry for NLLP 2024 shared task on Legal Natural Language Inference (L-NLI) The task required classifying relationships as entailed, contradicted, or neutral, indicating any association between the review and the complaint. Our system emerged as the winning submission, significantly outperforming other entries with a substantial margin and demonstrating the effectiveness of our approach in legal text analysis.
  arXiv  Detail & Related papers  (2024-10-21T13:20:15Z)
• LegalLens Shared Task 2024: Legal Violation Identification in Unstructured Text [7.839638824275218]
  This paper focuses on detecting legal violations within text in the wild across two sub-tasks: LegalLens-NER for identifying legal violation entities and LegalLens-NLI for associating these violations with relevant legal contexts and affected individuals. The top-performing team achieved a 7.11% improvement in NER over the baseline, while NLI saw a more marginal improvement of 5.7%.
  arXiv  Detail & Related papers  (2024-10-15T21:02:44Z)
• Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes [61.916827858666906]
  Large Language Models (LLMs) are becoming a prominent generative AI tool, where the user enters a query and the LLM generates an answer. To reduce harm and misuse, efforts have been made to align these LLMs to human values using advanced training techniques such as Reinforcement Learning from Human Feedback. Recent studies have highlighted the vulnerability of LLMs to adversarial jailbreak attempts aiming at subverting the embedded safety guardrails. This paper proposes a method called Gradient Cuff to detect jailbreak attempts.
  arXiv  Detail & Related papers  (2024-03-01T03:29:54Z)
• The Decisive Power of Indecision: Low-Variance Risk-Limiting Audits and Election Contestation via Marginal Mark Recording [51.82772358241505]
  Risk-limiting audits (RLAs) are techniques for verifying the outcomes of large elections. We define new families of audits that improve efficiency and offer advances in statistical power. New audits are enabled by revisiting the standard notion of a cast-vote record so that it can declare multiple possible mark interpretations.
  arXiv  Detail & Related papers  (2024-02-09T16:23:54Z)
• LegalLens: Leveraging LLMs for Legal Violation Identification in Unstructured Text [2.5218432691157866]
  We focus on two main tasks, the first for detecting legal violations within unstructured textual data, and the second for associating these violations with potentially affected individuals. We constructed two datasets using Large Language Models (LLMs) which were validated by domain expert annotators. Our results, with an F1-score of 62.69% (violation identification) and 81.02% (associating victims), show that our datasets and setups can be used for both tasks.
  arXiv  Detail & Related papers  (2024-02-06T19:18:56Z)
• A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction [60.70089334782383]
  Large language models (LLMs) have demonstrated great potential for domain-specific applications. Recent disputes over GPT-4's law evaluation raise questions concerning their performance in real-world legal tasks. We design practical baseline solutions based on LLMs and test on the task of legal judgment prediction.
  arXiv  Detail & Related papers  (2023-10-18T07:38:04Z)
• Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matching [67.98016412551245]
  We introduce a novel ASR system, ESPUM. This system harnesses the power of lower-order N-skipgrams (up to N=3) combined with positional unigram statistics gathered from a small batch of samples. Our model showcases competitive performance in ASR and phoneme segmentation tasks.
  arXiv  Detail & Related papers  (2023-10-03T19:05:32Z)
• Making Large Language Models Better Reasoners with Alignment [57.82176656663245]
  Reasoning is a cognitive process of using evidence to reach a sound conclusion. Recent studies reveal that fine-tuning LLMs on data with the chain of thought (COT) reasoning process can significantly enhance their reasoning capabilities. We introduce an textitAlignment Fine-Tuning (AFT) paradigm, which involves three steps.
  arXiv  Detail & Related papers  (2023-09-05T11:32:48Z)
• VL-NMS: Breaking Proposal Bottlenecks in Two-Stage Visual-Language Matching [75.71523183166799]
  The prevailing framework for matching multimodal inputs is based on a two-stage process. We argue that these methods overlook an obvious emphmismatch between the roles of proposals in the two stages. We propose VL-NMS, which is the first method to yield query-aware proposals at the first stage.
  arXiv  Detail & Related papers  (2021-05-12T13:05:25Z)
• A Legal Approach to Hate Speech: Operationalizing the EU's Legal Framework against the Expression of Hatred as an NLP Task [2.248133901806859]
  We propose a 'legal approach' to hate speech detection by operationalization of the decision as to whether a post is subject to criminal law. We show that, by breaking the legal assessment down into a series of simpler sub-decisions, even laypersons can annotate consistently.
  arXiv  Detail & Related papers  (2020-04-07T14:13:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.