Prototypical Representation Learning for Relation Extraction
- URL: http://arxiv.org/abs/2103.11647v1
- Date: Mon, 22 Mar 2021 08:11:43 GMT
- Title: Prototypical Representation Learning for Relation Extraction
- Authors: Ning Ding, Xiaobin Wang, Yao Fu, Guangwei Xu, Rui Wang, Pengjun Xie,
Ying Shen, Fei Huang, Hai-Tao Zheng, Rui Zhang
- Abstract summary: This paper aims to learn predictive, interpretable, and robust relation representations from distantly-labeled data.
We learn prototypes for each relation from contextual information to best explore the intrinsic semantics of relations.
Results on several relation learning tasks show that our model significantly outperforms the previous state-of-the-art relational models.
- Score: 56.501332067073065
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recognizing relations between entities is a pivotal task of relational
learning. Learning relation representations from distantly-labeled datasets is
difficult because of the abundant label noise and complicated expressions in
human language. This paper aims to learn predictive, interpretable, and robust
relation representations from distantly-labeled data that are effective in
different settings, including supervised, distantly supervised, and few-shot
learning. Instead of solely relying on the supervision from noisy labels, we
propose to learn prototypes for each relation from contextual information to
best explore the intrinsic semantics of relations. Prototypes are
representations in the feature space abstracting the essential semantics of
relations between entities in sentences. We learn prototypes based on
objectives with clear geometric interpretation, where the prototypes are unit
vectors uniformly dispersed in a unit ball, and statement embeddings are
centered at the end of their corresponding prototype vectors on the surface of
the ball. This approach allows us to learn meaningful, interpretable prototypes
for the final classification. Results on several relation learning tasks show
that our model significantly outperforms the previous state-of-the-art models.
We further demonstrate the robustness of the encoder and the interpretability
of prototypes with extensive experiments.
Related papers
- Simple-Sampling and Hard-Mixup with Prototypes to Rebalance Contrastive Learning for Text Classification [11.072083437769093]
We propose a novel model named SharpReCL for imbalanced text classification tasks.
Our model even outperforms popular large language models across several datasets.
arXiv Detail & Related papers (2024-05-19T11:33:49Z) - Corpus Considerations for Annotator Modeling and Scaling [9.263562546969695]
We show that the commonly used user token model consistently outperforms more complex models.
Our findings shed light on the relationship between corpus statistics and annotator modeling performance.
arXiv Detail & Related papers (2024-04-02T22:27:24Z) - Prototype-based Embedding Network for Scene Graph Generation [105.97836135784794]
Current Scene Graph Generation (SGG) methods explore contextual information to predict relationships among entity pairs.
Due to the diverse visual appearance of numerous possible subject-object combinations, there is a large intra-class variation within each predicate category.
Prototype-based Embedding Network (PE-Net) models entities/predicates with prototype-aligned compact and distinctive representations.
PL is introduced to help PE-Net efficiently learn such entitypredicate matching, and Prototype Regularization (PR) is devised to relieve the ambiguous entity-predicate matching.
arXiv Detail & Related papers (2023-03-13T13:30:59Z) - The Trade-off between Universality and Label Efficiency of
Representations from Contrastive Learning [32.15608637930748]
We show that there exists a trade-off between the two desiderata so that one may not be able to achieve both simultaneously.
We provide analysis using a theoretical data model and show that, while more diverse pre-training data result in more diverse features for different tasks, it puts less emphasis on task-specific features.
arXiv Detail & Related papers (2023-02-28T22:14:33Z) - Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
We tackle the problem of learning visual representations from unlabeled scene-centric data.
We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
arXiv Detail & Related papers (2022-05-30T17:50:59Z) - A Simple yet Effective Relation Information Guided Approach for Few-Shot
Relation Extraction [22.60428265210431]
Few-Shot Relation Extraction aims at predicting the relation for a pair of entities in a sentence by training with a few labelled examples in each relation.
Some recent works have introduced relation information to assist model learning based on Prototype Network.
We argue that relation information can be introduced more explicitly and effectively into the model.
arXiv Detail & Related papers (2022-05-19T13:03:01Z) - Learning Relation Prototype from Unlabeled Texts for Long-tail Relation
Extraction [84.64435075778988]
We propose a general approach to learn relation prototypes from unlabeled texts.
We learn relation prototypes as an implicit factor between entities.
We conduct experiments on two publicly available datasets: New York Times and Google Distant Supervision.
arXiv Detail & Related papers (2020-11-27T06:21:12Z) - Relation-Guided Representation Learning [53.60351496449232]
We propose a new representation learning method that explicitly models and leverages sample relations.
Our framework well preserves the relations between samples.
By seeking to embed samples into subspace, we show that our method can address the large-scale and out-of-sample problem.
arXiv Detail & Related papers (2020-07-11T10:57:45Z) - Prototypical Contrastive Learning of Unsupervised Representations [171.3046900127166]
Prototypical Contrastive Learning (PCL) is an unsupervised representation learning method.
PCL implicitly encodes semantic structures of the data into the learned embedding space.
PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks.
arXiv Detail & Related papers (2020-05-11T09:53:36Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.