Generative Entity Typing with Curriculum Learning
- URL: http://arxiv.org/abs/2210.02914v1
- Date: Thu, 6 Oct 2022 13:32:50 GMT
- Title: Generative Entity Typing with Curriculum Learning
- Authors: Siyu Yuan, Deqing Yang, Jiaqing Liang, Zhixu Li, Jinxi Liu, Jingyue
Huang, Yanghua Xiao
- Abstract summary: We propose a novel generative entity typing (GET) paradigm.
Given a text with an entity mention, the multiple types for the role that the entity plays in the text are generated with a pre-trained language model.
Our experiments justify the superiority of our GET model over the state-of-the-art entity typing models.
- Score: 18.43562065432877
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Entity typing aims to assign types to the entity mentions in given texts. The
traditional classification-based entity typing paradigm has two unignorable
drawbacks: 1) it fails to assign an entity to the types beyond the predefined
type set, and 2) it can hardly handle few-shot and zero-shot situations where
many long-tail types only have few or even no training instances. To overcome
these drawbacks, we propose a novel generative entity typing (GET) paradigm:
given a text with an entity mention, the multiple types for the role that the
entity plays in the text are generated with a pre-trained language model (PLM).
However, PLMs tend to generate coarse-grained types after fine-tuning upon the
entity typing dataset. Besides, we only have heterogeneous training data
consisting of a small portion of human-annotated data and a large portion of
auto-generated but low-quality data. To tackle these problems, we employ
curriculum learning (CL) to train our GET model upon the heterogeneous data,
where the curriculum could be self-adjusted with the self-paced learning
according to its comprehension of the type granularity and data heterogeneity.
Our extensive experiments upon the datasets of different languages and
downstream tasks justify the superiority of our GET model over the
state-of-the-art entity typing models. The code has been released on
https://github.com/siyuyuan/GET.
Related papers
- Language Models for Text Classification: Is In-Context Learning Enough? [54.869097980761595]
Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings.
An advantage of these models over more standard approaches is the ability to understand instructions written in natural language (prompts)
This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances.
arXiv Detail & Related papers (2024-03-26T12:47:39Z) - Seed-Guided Fine-Grained Entity Typing in Science and Engineering
Domains [51.02035914828596]
We study the task of seed-guided fine-grained entity typing in science and engineering domains.
We propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus.
It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types.
arXiv Detail & Related papers (2024-01-23T22:36:03Z) - From Ultra-Fine to Fine: Fine-tuning Ultra-Fine Entity Typing Models to
Fine-grained [12.948753628039093]
A common way to address this problem is to use distantly annotated training data that contains incorrect labels.
We propose a new approach that can avoid the need of creating distantly labeled data whenever there is a new type schema.
arXiv Detail & Related papers (2023-12-11T08:12:01Z) - Ontology Enrichment for Effective Fine-grained Entity Typing [45.356694904518626]
Fine-grained entity typing (FET) is the task of identifying specific entity types at a fine-grained level for entity mentions based on their contextual information.
Conventional methods for FET require extensive human annotation, which is time-consuming and costly.
We develop a coarse-to-fine typing algorithm that exploits the enriched information by training an entailment model with contrasting topics and instance-based augmented training samples.
arXiv Detail & Related papers (2023-10-11T18:30:37Z) - OntoType: Ontology-Guided and Pre-Trained Language Model Assisted Fine-Grained Entity Typing [25.516304052884397]
Fine-grained entity typing (FET) assigns entities in text with context-sensitive, fine-grained semantic types.
OntoType follows a type ontological structure, from coarse to fine, ensembles multiple PLM prompting results to generate a set of type candidates.
Our experiments on the Ontonotes, FIGER, and NYT datasets demonstrate that our method outperforms the state-of-the-art zero-shot fine-grained entity typing methods.
arXiv Detail & Related papers (2023-05-21T00:32:37Z) - Few-Shot Fine-Grained Entity Typing with Automatic Label Interpretation
and Instance Generation [36.541309948222306]
We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type.
We propose a novel framework for few-shot FET consisting of two modules: (1) an entity type label interpretation module automatically learns to relate type labels to the vocabulary by jointly leveraging few-shot instances and the label hierarchy, and (2) a type-based contextualized instance generator produces new instances based on given instances to enlarge the training set for better generalization.
arXiv Detail & Related papers (2022-06-28T04:05:40Z) - Learning Contextual Representations for Semantic Parsing with
Generation-Augmented Pre-Training [86.91380874390778]
We present Generation-Augmented Pre-training (GAP), that jointly learns representations of natural language utterances and table schemas by leveraging generation models to generate pre-train data.
Based on experimental results, neural semantics that leverage GAP MODEL obtain new state-of-the-art results on both SPIDER and CRITERIA-TO-generative benchmarks.
arXiv Detail & Related papers (2020-12-18T15:53:50Z) - Unsupervised Paraphrasing with Pretrained Language Models [85.03373221588707]
We propose a training pipeline that enables pre-trained language models to generate high-quality paraphrases in an unsupervised setting.
Our recipe consists of task-adaptation, self-supervision, and a novel decoding algorithm named Dynamic Blocking.
We show with automatic and human evaluations that our approach achieves state-of-the-art performance on both the Quora Question Pair and the ParaNMT datasets.
arXiv Detail & Related papers (2020-10-24T11:55:28Z) - Interpretable Entity Representations through Large-Scale Typing [61.4277527871572]
We present an approach to creating entity representations that are human readable and achieve high performance out of the box.
Our representations are vectors whose values correspond to posterior probabilities over fine-grained entity types.
We show that it is possible to reduce the size of our type set in a learning-based way for particular domains.
arXiv Detail & Related papers (2020-04-30T23:58:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.