WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web

• URL: http://arxiv.org/abs/2102.09681v1
• Date: Thu, 18 Feb 2021 23:56:12 GMT
• Title: WebRED: Effective Pretraining And Finetuning For Relation Extraction On The Web
• Authors: Robert Ormandi, Mohammad Saleh, Erin Winter, Vinay Rao
• Abstract summary: WebRED is a strongly-supervised human annotated dataset for extracting relationships from text found on the World Wide Web. We show that combining pre-training on a large weakly supervised dataset with fine-tuning on a small strongly-supervised dataset leads to better relation extraction performance.
• Score: 4.702325864333419
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Relation extraction is used to populate knowledge bases that are important to many applications. Prior datasets used to train relation extraction models either suffer from noisy labels due to distant supervision, are limited to certain domains or are too small to train high-capacity models. This constrains downstream applications of relation extraction. We therefore introduce: WebRED (Web Relation Extraction Dataset), a strongly-supervised human annotated dataset for extracting relationships from a variety of text found on the World Wide Web, consisting of ~110K examples. We also describe the methods we used to collect ~200M examples as pre-training data for this task. We show that combining pre-training on a large weakly supervised dataset with fine-tuning on a small strongly-supervised dataset leads to better relation extraction performance. We provide baselines for this new dataset and present a case for the importance of human annotation in improving the performance of relation extraction from text found on the web.

Related papers

• STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models [56.27786433792638]
  STAR is a data generation method that leverages Large Language Models (LLMs) to synthesize data instances. We design fine-grained step-by-step instructions to obtain the initial data instances. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks.
  arXiv  Detail & Related papers  (2023-05-24T12:15:19Z)
• Beyond Transfer Learning: Co-finetuning for Action Localisation [64.07196901012153]
  We propose co-finetuning -- simultaneously training a single model on multiple upstream'' and downstream'' tasks. We demonstrate that co-finetuning outperforms traditional transfer learning when using the same total amount of data. We also show how we can easily extend our approach to multiple upstream'' datasets to further improve performance.
  arXiv  Detail & Related papers  (2022-07-08T10:25:47Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• FREDA: Flexible Relation Extraction Data Annotation [1.3750624267664153]
  We propose an approach to produce high-quality datasets for the task of Relation Extraction quickly. In our study, we were able to annotate 10,022 sentences for 19 relations in a reasonable amount of time.
  arXiv  Detail & Related papers  (2022-04-14T17:57:53Z)
• RTE: A Tool for Annotating Relation Triplets from Text [3.2958527541557525]
  In relation extraction, we focus on binary relation that refers to relations between two entities. The lack of annotated clean dataset is a key challenge in this area of research. In this work, we built a web-based tool where researchers can annotate for relation extraction on their own datasets.
  arXiv  Detail & Related papers  (2021-08-18T14:54:22Z)
• Representation Learning for Weakly Supervised Relation Extraction [19.689433249830465]
  In this thesis, we present several novel unsupervised pre-training models to learn the distributed text representation features. The experiments have demonstrated that this type of feature, combine with the traditional hand-crafted features, could improve the performance of the logistic classification model for relation extraction.
  arXiv  Detail & Related papers  (2021-04-10T12:22:25Z)
• Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey [22.586079965178975]
  We focus on relation extraction using deep neural networks on publicly available datasets. We cover sentence-level relation extraction to document-level relation extraction, pipeline-based approaches to joint extraction approaches, annotated datasets to distantly supervised datasets. Regarding neural architectures, we cover convolutional models, recurrent network models, attention network models, and graph convolutional models in this survey.
  arXiv  Detail & Related papers  (2021-03-31T09:27:15Z)
• Relation-Guided Representation Learning [53.60351496449232]
  We propose a new representation learning method that explicitly models and leverages sample relations. Our framework well preserves the relations between samples. By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
  arXiv  Detail & Related papers  (2020-07-11T10:57:45Z)
• Learning to Count in the Crowd from Limited Labeled Data [109.2954525909007]
  We focus on reducing the annotation efforts by learning to count in the crowd from limited number of labeled samples. Specifically, we propose a Gaussian Process-based iterative learning mechanism that involves estimation of pseudo-ground truth for the unlabeled data.
  arXiv  Detail & Related papers  (2020-07-07T04:17:01Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.