Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation

• URL: http://arxiv.org/abs/2203.10900v1
• Date: Mon, 21 Mar 2022 11:48:40 GMT
• Title: Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation
• Authors: Qingyu Tan, Ruidan He, Lidong Bing, Hwee Tou Ng
• Abstract summary: Document-level Relation Extraction (DocRE) is a more challenging task compared to its sentence-level counterpart. We propose a semi-supervised framework for DocRE with three novel components.
• Score: 44.39395368597772
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Document-level Relation Extraction (DocRE) is a more challenging task compared to its sentence-level counterpart. It aims to extract relations from multiple sentences at once. In this paper, we propose a semi-supervised framework for DocRE with three novel components. Firstly, we use an axial attention module for learning the interdependency among entity-pairs, which improves the performance on two-hop relations. Secondly, we propose an adaptive focal loss to tackle the class imbalance problem of DocRE. Lastly, we use knowledge distillation to overcome the differences between human annotated data and distantly supervised data. We conducted experiments on two DocRE datasets. Our model consistently outperforms strong baselines and its performance exceeds the previous SOTA by 1.36 F1 and 1.46 Ign_F1 score on the DocRED leaderboard. Our code and data will be released at https://github.com/tonytan48/KD-DocRE.

Related papers

• DocMamba: Efficient Document Pre-training with State Space Model [56.84200017560988]
  We present DocMamba, a novel framework based on the state space model. It is designed to reduce computational complexity to linear while preserving global modeling capabilities. Experiments on the HRDoc confirm DocMamba's potential for length extrapolation.
  arXiv  Detail & Related papers  (2024-09-18T11:34:28Z)
• Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction [43.50683283748675]
  Document-level Relation Triplet Extraction (DocRTE) is a fundamental task in information systems that aims to simultaneously extract entities with semantic relations from a document. Existing methods heavily rely on a substantial amount of fully labeled data. Recent advanced Large Language Models (LLMs), such as ChatGPT and LLaMA, exhibit impressive long-text generation capabilities.
  arXiv  Detail & Related papers  (2024-01-24T17:04:28Z)
• Document-Level In-Context Few-Shot Relation Extraction via Pre-Trained Language Models [29.94694305204144]
  We present a novel framework for document-level in-context few-shot relation extraction. We evaluate our framework using DocRED, the largest publicly available dataset for document-level relation extraction.
  arXiv  Detail & Related papers  (2023-10-17T09:10:27Z)
• PRiSM: Enhancing Low-Resource Document-Level Relation Extraction with Relation-Aware Score Calibration [44.074482478955126]
  Document-level relation extraction (DocRE) aims to extract relations of all entity pairs in a document. Key challenge in DocRE is the cost of annotating such data which requires intensive human effort. We propose PRiSM, which learns to adapt logits based on relation semantic information.
  arXiv  Detail & Related papers  (2023-09-25T04:42:39Z)
• Mutually Guided Few-shot Learning for Relational Triple Extraction [10.539566491939844]
  Mutually Guided Few-shot learning framework for Triple Extraction (MG-FTE) Our method consists of an entity-guided relation-decoder to classify relations and a proto-decoder to extract entities. Our method outperforms many state-of-the-art methods by 12.6 F1 score on FewRel 1.0 (single domain) and 20.5 F1 score on FewRel 2.0 (cross-domain)
  arXiv  Detail & Related papers  (2023-06-23T06:15:54Z)
• HIORE: Leveraging High-order Interactions for Unified Entity Relation Extraction [85.80317530027212]
  We propose HIORE, a new method for unified entity relation extraction. The key insight is to leverage the complex association among word pairs, which contains richer information than the first-order word-by-word interactions. Experiments show that HIORE achieves the state-of-the-art performance on relation extraction and an improvement of 1.11.8 F1 points over the prior best unified model.
  arXiv  Detail & Related papers  (2023-05-07T14:57:42Z)
• Document-Level Relation Extraction with Sentences Importance Estimation and Focusing [52.069206266557266]
  Document-level relation extraction (DocRE) aims to determine the relation between two entities from a document of multiple sentences. We propose a Sentence Estimation and Focusing (SIEF) framework for DocRE, where we design a sentence importance score and a sentence focusing loss. Experimental results on two domains show that our SIEF not only improves overall performance, but also makes DocRE models more robust.
  arXiv  Detail & Related papers  (2022-04-27T03:20:07Z)
• SAIS: Supervising and Augmenting Intermediate Steps for Document-Level Relation Extraction [51.27558374091491]
  We propose to explicitly teach the model to capture relevant contexts and entity types by supervising and augmenting intermediate steps (SAIS) for relation extraction. Based on a broad spectrum of carefully designed tasks, our proposed SAIS method not only extracts relations of better quality due to more effective supervision, but also retrieves the corresponding supporting evidence more accurately.
  arXiv  Detail & Related papers  (2021-09-24T17:37:35Z)
• Eider: Evidence-enhanced Document-level Relation Extraction [56.71004595444816]
  Document-level relation extraction (DocRE) aims at extracting semantic relations among entity pairs in a document. We propose a three-stage evidence-enhanced DocRE framework consisting of joint relation and evidence extraction, evidence-centered relation extraction (RE), and fusion of extraction results.
  arXiv  Detail & Related papers  (2021-06-16T09:43:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.