MARRO: Multi-headed Attention for Rhetorical Role Labeling in Legal Documents

• URL: http://arxiv.org/abs/2503.10659v1
• Date: Sat, 08 Mar 2025 08:05:20 GMT
• Title: MARRO: Multi-headed Attention for Rhetorical Role Labeling in Legal Documents
• Authors: Purbid Bambroo, Subinay Adhikary, Paheli Bhattacharya, Abhijnan Chakraborty, Saptarshi Ghosh, Kripabandhu Ghosh,
• Abstract summary: Identification of rhetorical roles like facts, arguments, and final judgments is central to understanding a legal case document.<n>Legal documents are often unstructured and contain a specialized vocabulary, making it hard for conventional transformer models to understand them.<n>We propose a novel family of multi-task learning-based models for rhetorical role labeling, named MARRO, that uses transformer-inspired multi-headed attention.
• Score: 8.596233578884162
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Identification of rhetorical roles like facts, arguments, and final judgments is central to understanding a legal case document and can lend power to other downstream tasks like legal case summarization and judgment prediction. However, there are several challenges to this task. Legal documents are often unstructured and contain a specialized vocabulary, making it hard for conventional transformer models to understand them. Additionally, these documents run into several pages, which makes it difficult for neural models to capture the entire context at once. Lastly, there is a dearth of annotated legal documents to train deep learning models. Previous state-of-the-art approaches for this task have focused on using neural models like BiLSTM-CRF or have explored different embedding techniques to achieve decent results. While such techniques have shown that better embedding can result in improved model performance, not many models have focused on utilizing attention for learning better embeddings in sentences of a document. Additionally, it has been recently shown that advanced techniques like multi-task learning can help the models learn better representations, thereby improving performance. In this paper, we combine these two aspects by proposing a novel family of multi-task learning-based models for rhetorical role labeling, named MARRO, that uses transformer-inspired multi-headed attention. Using label shift as an auxiliary task, we show that models from the MARRO family achieve state-of-the-art results on two labeled datasets for rhetorical role labeling, from the Indian and UK Supreme Courts.

Related papers

• LegalSeg: Unlocking the Structure of Indian Legal Judgments Through Rhetorical Role Classification [6.549338652948716]
  We introduce LegalSeg, the largest annotated dataset for this task, comprising over 7,000 documents and 1.4 million sentences, labeled with 7 rhetorical roles.<n>Our results demonstrate that models incorporating broader context, structural relationships, and sequential sentence information outperform those relying solely on sentence-level features.
  arXiv  Detail & Related papers  (2025-02-09T10:07:05Z)
• VEGA: Learning Interleaved Image-Text Comprehension in Vision-Language Large Models [76.94378391979228]
  We introduce a new, more demanding task known as Interleaved Image-Text (IITC) This task challenges models to discern and disregard superfluous elements in both images and text to accurately answer questions. In support of this task, we further craft a new VEGA dataset, tailored for the IITC task on scientific content, and devised a subtask, Image-Text Association (ITA)
  arXiv  Detail & Related papers  (2024-06-14T17:59:40Z)
• Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard [47.73060223236792]
  BEIR is a benchmark dataset for evaluation of information retrieval models across 18 different domain/task combinations. Our work addresses two shortcomings that prevent the benchmark from achieving its full potential.
  arXiv  Detail & Related papers  (2023-06-13T00:26:18Z)
• Rhetorical Role Labeling of Legal Documents using Transformers and Graph Neural Networks [1.290382979353427]
  This paper presents the approaches undertaken to perform the task of rhetorical role labelling on Indian Court Judgements as part of SemEval Task 6: understanding legal texts, shared subtask A.
  arXiv  Detail & Related papers  (2023-05-06T17:04:51Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Semantic Segmentation of Legal Documents via Rhetorical Roles [3.285073688021526]
  This paper proposes a Rhetorical Roles (RR) system for segmenting a legal document into semantically coherent units. We develop a multitask learning-based deep learning model with document rhetorical role label shift as an auxiliary task for segmenting a legal document.
  arXiv  Detail & Related papers  (2021-12-03T10:49:19Z)
• TAN-NTM: Topic Attention Networks for Neural Topic Modeling [8.631228373008478]
  We propose a novel framework: TAN-NTM which models document as a sequence of tokens instead of BoW at the input layer. We apply attention on LSTM outputs to empower the model to attend on relevant words which convey topic related cues. TAN-NTM achieves state-of-the-art results with 9-15 percentage improvement over score of existing SOTA topic models in NPMI coherence metric.
  arXiv  Detail & Related papers  (2020-12-02T20:58:04Z)
• Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling [81.33107307509718]
  We propose a topic adaptive storyteller to model the ability of inter-topic generalization. We also propose a prototype encoding structure to model the ability of intra-topic derivation. Experimental results show that topic adaptation and prototype encoding structure mutually bring benefit to the few-shot model.
  arXiv  Detail & Related papers  (2020-08-11T03:55:11Z)
• Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension [127.3341842928421]
  Natural Questions is a new challenging machine reading comprehension benchmark. It has two-grained answers, which are a long answer (typically a paragraph) and a short answer (one or more entities inside the long answer) Existing methods treat these two sub-tasks individually during training while ignoring their dependencies. We present a novel multi-grained machine reading comprehension framework that focuses on modeling documents at their hierarchical nature.
  arXiv  Detail & Related papers  (2020-05-12T14:20:09Z)
• SPECTER: Document-level Representation Learning using Citation-informed Transformers [51.048515757909215]
  SPECTER generates document-level embedding of scientific documents based on pretraining a Transformer language model. We introduce SciDocs, a new evaluation benchmark consisting of seven document-level tasks ranging from citation prediction to document classification and recommendation.
  arXiv  Detail & Related papers  (2020-04-15T16:05:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.