Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model

• URL: http://arxiv.org/abs/2106.04098v1
• Date: Tue, 8 Jun 2021 04:43:28 GMT
• Title: Ultra-Fine Entity Typing with Weak Supervision from a Masked Language Model
• Authors: Hongliang Dai, Yangqiu Song, Haixun Wang
• Abstract summary: Recently there is an effort to extend fine-grained entity typing by using a richer and ultra-fine set of types. We propose to obtain training data for ultra-fine entity typing by using a BERT Masked Language Model (MLM) Given a mention in a sentence, our approach constructs an input for the BERT so that it predicts context dependent hypernyms of the mention, which can be used as type labels.
• Score: 39.031515304057585
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, there is an effort to extend fine-grained entity typing by using a richer and ultra-fine set of types, and labeling noun phrases including pronouns and nominal nouns instead of just named entity mentions. A key challenge for this ultra-fine entity typing task is that human annotated data are extremely scarce, and the annotation ability of existing distant or weak supervision approaches is very limited. To remedy this problem, in this paper, we propose to obtain training data for ultra-fine entity typing by using a BERT Masked Language Model (MLM). Given a mention in a sentence, our approach constructs an input for the BERT MLM so that it predicts context dependent hypernyms of the mention, which can be used as type labels. Experimental results demonstrate that, with the help of these automatically generated labels, the performance of an ultra-fine entity typing model can be improved substantially. We also show that our approach can be applied to improve traditional fine-grained entity typing after performing simple type mapping.

Related papers

• Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels [75.77877889764073]
  Large Language Models (LLMs) have demonstrated remarkable performance through supervised fine-tuning or in-context learning using gold labels. This study explores whether solely utilizing unlabeled data can elicit strong model capabilities. We propose a new paradigm termed zero-to-strong generalization.
  arXiv  Detail & Related papers  (2024-09-19T02:59:44Z)
• Seed-Guided Fine-Grained Entity Typing in Science and Engineering Domains [51.02035914828596]
  We study the task of seed-guided fine-grained entity typing in science and engineering domains. We propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus. It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types.
  arXiv  Detail & Related papers  (2024-01-23T22:36:03Z)
• Which Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code? [51.29970742152668]
  We highlight relying on accuracy-based measurements may lead to an overestimation of models' capabilities. To address these issues, we introduce a technique called SyntaxEval in Syntactic Capabilities.
  arXiv  Detail & Related papers  (2024-01-03T02:44:02Z)
• From Ultra-Fine to Fine: Fine-tuning Ultra-Fine Entity Typing Models to Fine-grained [12.948753628039093]
  A common way to address this problem is to use distantly annotated training data that contains incorrect labels. We propose a new approach that can avoid the need of creating distantly labeled data whenever there is a new type schema.
  arXiv  Detail & Related papers  (2023-12-11T08:12:01Z)
• Ground Truth Inference for Weakly Supervised Entity Matching [76.6732856489872]
  We propose a simple but powerful labeling model for weak supervision tasks. We then tailor the labeling model specifically to the task of entity matching. We show that our labeling model results in a 9% higher F1 score on average than the best existing method.
  arXiv  Detail & Related papers  (2022-11-13T17:57:07Z)
• Generative Entity Typing with Curriculum Learning [18.43562065432877]
  We propose a novel generative entity typing (GET) paradigm. Given a text with an entity mention, the multiple types for the role that the entity plays in the text are generated with a pre-trained language model. Our experiments justify the superiority of our GET model over the state-of-the-art entity typing models.
  arXiv  Detail & Related papers  (2022-10-06T13:32:50Z)
• Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation and Instance Generation [36.541309948222306]
  We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type. We propose a novel framework for few-shot FET consisting of two modules: (1) an entity type label interpretation module automatically learns to relate type labels to the vocabulary by jointly leveraging few-shot instances and the label hierarchy, and (2) a type-based contextualized instance generator produces new instances based on given instances to enlarge the training set for better generalization.
  arXiv  Detail & Related papers  (2022-06-28T04:05:40Z)
• Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference [28.78215056129358]
  This work presents LITE, a new approach that formulates entity typing as a natural language inference (NLI) problem. Experiments show that, with limited training data, LITE obtains state-of-the-art performance on the UFET task.
  arXiv  Detail & Related papers  (2022-02-12T23:56:26Z)
• Interpretable Entity Representations through Large-Scale Typing [61.4277527871572]
  We present an approach to creating entity representations that are human readable and achieve high performance out of the box. Our representations are vectors whose values correspond to posterior probabilities over fine-grained entity types. We show that it is possible to reduce the size of our type set in a learning-based way for particular domains.
  arXiv  Detail & Related papers  (2020-04-30T23:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.