Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings

• URL: http://arxiv.org/abs/2102.01156v1
• Date: Mon, 1 Feb 2021 20:50:24 GMT
• Title: Improving Distantly-Supervised Relation Extraction through BERT-based Label & Instance Embeddings
• Authors: Despina Christou, Grigorios Tsoumakas
• Abstract summary: We propose REDSandT, a novel distantly-supervised transformer-based RE method. We exploit BERT's pre-trained model, and the relationship between labels and entities, respectively. Experiments in the NYT-10 dataset show that REDSandT captures a broader set of relations with higher confidence.
• Score: 2.88848244747161
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Distantly-supervised relation extraction (RE) is an effective method to scale RE to large corpora but suffers from noisy labels. Existing approaches try to alleviate noise through multi-instance learning and by providing additional information, but manage to recognize mainly the top frequent relations, neglecting those in the long-tail. We propose REDSandT (Relation Extraction with Distant Supervision and Transformers), a novel distantly-supervised transformer-based RE method, that manages to capture a wider set of relations through highly informative instance and label embeddings for RE, by exploiting BERT's pre-trained model, and the relationship between labels and entities, respectively. We guide REDSandT to focus solely on relational tokens by fine-tuning BERT on a structured input, including the sub-tree connecting an entity pair and the entities' types. Using the extracted informative vectors, we shape label embeddings, which we also use as attention mechanism over instances to further reduce noise. Finally, we represent sentences by concatenating relation and instance embeddings. Experiments in the NYT-10 dataset show that REDSandT captures a broader set of relations with higher confidence, achieving state-of-the-art AUC (0.424).

Related papers

• Distantly-Supervised Joint Extraction with Noise-Robust Learning [36.23022433465051]
  We focus on the problem of joint extraction in distantly-labeled data, whose labels are generated by aligning entity mentions with the corresponding entity and relation tags using a knowledge base (KB) Existing approaches, either considering only one source of noise or making decisions using external knowledge, cannot well-utilize significant information in the training data. We propose DENRL, a generalizable framework that incorporates a lightweight transformer backbone into a sequence labeling scheme for joint tagging.
  arXiv  Detail & Related papers  (2023-10-08T03:42:15Z)
• Mutually Guided Few-shot Learning for Relational Triple Extraction [10.539566491939844]
  Mutually Guided Few-shot learning framework for Triple Extraction (MG-FTE) Our method consists of an entity-guided relation-decoder to classify relations and a proto-decoder to extract entities. Our method outperforms many state-of-the-art methods by 12.6 F1 score on FewRel 1.0 (single domain) and 20.5 F1 score on FewRel 2.0 (cross-domain)
  arXiv  Detail & Related papers  (2023-06-23T06:15:54Z)
• RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extraction [65.4337085607711]
  We introduce the task setting of Zero-Shot Relation Triplet Extraction (ZeroRTE) Given an input sentence, each extracted triplet consists of the head entity, relation label, and tail entity where the relation label is not seen at the training stage. We propose to synthesize relation examples by prompting language models to generate structured texts.
  arXiv  Detail & Related papers  (2022-03-17T05:55:14Z)
• AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction [54.20039200180071]
  We present a novel framework to improve NER performance by automatically generating and leveraging entity triggers'' Our framework leverages post-hoc explanation to generate rationales and strengthens a model's prior knowledge using an embedding technique. AutoTriggER shows strong label-efficiency, is capable of generalizing to unseen entities, and outperforms the RoBERTa-CRF baseline by nearly 0.5 F1 points on average.
  arXiv  Detail & Related papers  (2021-09-10T08:11:56Z)
• An Embarrassingly Simple Model for Dialogue Relation Extraction [44.2379205657313]
  We model Dialogue RE as a multi-label classification task and propose a simple yet effective model named SimpleRE. SimpleRE captures the interrelations among multiple relations in a dialogue through a novel input format, BERT Relation Token Sequence (BRS) A Relation Refinement Gate (RRG) is designed to extract relation-specific semantic representation adaptively.
  arXiv  Detail & Related papers  (2020-12-27T06:22:23Z)
• Relation Clustering in Narrative Knowledge Graphs [71.98234178455398]
  relational sentences in the original text are embedded (with SBERT) and clustered in order to merge together semantically similar relations. Preliminary tests show that such clustering might successfully detect similar relations, and provide a valuable preprocessing for semi-supervised approaches.
  arXiv  Detail & Related papers  (2020-11-27T10:43:04Z)
• Learning to Decouple Relations: Few-Shot Relation Classification with Entity-Guided Attention and Confusion-Aware Training [49.9995628166064]
  We propose CTEG, a model equipped with two mechanisms to learn to decouple easily-confused relations. On the one hand, an EGA mechanism is introduced to guide the attention to filter out information causing confusion. On the other hand, a Confusion-Aware Training (CAT) method is proposed to explicitly learn to distinguish relations.
  arXiv  Detail & Related papers  (2020-10-21T11:07:53Z)
• Cross-Supervised Joint-Event-Extraction with Heterogeneous Information Networks [61.950353376870154]
  Joint-event-extraction is a sequence-to-sequence labeling task with a tag set composed of tags of triggers and entities. We propose a Cross-Supervised Mechanism (CSM) to alternately supervise the extraction of triggers or entities. Our approach outperforms the state-of-the-art methods in both entity and trigger extraction.
  arXiv  Detail & Related papers  (2020-10-13T11:51:17Z)
• Distantly-Supervised Neural Relation Extraction with Side Information using BERT [2.0946724304757955]
  Relation extraction (RE) consists in categorizing the relationship between entities in a sentence. One of the methods that adopt this strategy is the RESIDE model, which proposes a distantly-supervised neural relation extraction using side information from Knowledge Bases. Considering that this method outperformed state-of-the-art baselines, in this paper, we propose a related approach to RESIDE also using additional side information, but simplifying the sentence encoding with BERT embeddings.
  arXiv  Detail & Related papers  (2020-04-29T19:29:10Z)
• Relabel the Noise: Joint Extraction of Entities and Relations via Cooperative Multiagents [52.55119217982361]
  We propose a joint extraction approach to handle noisy instances with a group of cooperative multiagents. To handle noisy instances in a fine-grained manner, each agent in the cooperative group evaluates the instance by calculating a continuous confidence score from its own perspective. A confidence consensus module is designed to gather the wisdom of all agents and re-distribute the noisy training set with confidence-scored labels.
  arXiv  Detail & Related papers  (2020-04-21T12:03:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.