Modeling Fine-Grained Entity Types with Box Embeddings
- URL: http://arxiv.org/abs/2101.00345v1
- Date: Sat, 2 Jan 2021 00:59:10 GMT
- Title: Modeling Fine-Grained Entity Types with Box Embeddings
- Authors: Yasumasa Onoe, Michael Boratko, Greg Durrett
- Abstract summary: We study the ability of box embeddings to represent hierarchies of fine-grained entity type labels.
We compare our approach with a strong vector-based typing model, and observe state-of-the-art performance on several entity typing benchmarks.
- Score: 32.85605894725522
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Neural entity typing models typically represent entity types as vectors in a
high-dimensional space, but such spaces are not well-suited to modeling these
types' complex interdependencies. We study the ability of box embeddings, which
represent entity types as d-dimensional hyperrectangles, to represent
hierarchies of fine-grained entity type labels even when these relationships
are not defined explicitly in the ontology. Our model represents both types and
entity mentions as boxes. Each mention and its context are fed into a
BERT-based model to embed that mention in our box space; essentially, this
model leverages typological clues present in the surface text to hypothesize a
type representation for the mention. Soft box containment can then be used to
derive probabilities, both the posterior probability of a mention exhibiting a
given type and the conditional probability relations between types themselves.
We compare our approach with a strong vector-based typing model, and observe
state-of-the-art performance on several entity typing benchmarks. In addition
to competitive typing performance, our box-based model shows better performance
in prediction consistency (predicting a supertype and a subtype together) and
confidence (i.e., calibration), implying that the box-based model captures the
latent type hierarchies better than the vector-based model does.
Related papers
- Seed-Guided Fine-Grained Entity Typing in Science and Engineering
Domains [51.02035914828596]
We study the task of seed-guided fine-grained entity typing in science and engineering domains.
We propose SEType which first enriches the weak supervision by finding more entities for each seen type from an unlabeled corpus.
It then matches the enriched entities to unlabeled text to get pseudo-labeled samples and trains a textual entailment model that can make inferences for both seen and unseen types.
arXiv Detail & Related papers (2024-01-23T22:36:03Z) - Meaning Representations from Trajectories in Autoregressive Models [106.63181745054571]
We propose to extract meaning representations from autoregressive language models by considering the distribution of all possible trajectories extending an input text.
This strategy is prompt-free, does not require fine-tuning, and is applicable to any pre-trained autoregressive model.
We empirically show that the representations obtained from large models align well with human annotations, outperform other zero-shot and prompt-free methods on semantic similarity tasks, and can be used to solve more complex entailment and containment tasks that standard embeddings cannot handle.
arXiv Detail & Related papers (2023-10-23T04:35:58Z) - Polar Ducks and Where to Find Them: Enhancing Entity Linking with Duck
Typing and Polar Box Embeddings [11.164501311947124]
DUCK is an approach to infusing structural information in the space of entity representations.
We define the type of an entity based on the relations that it has with other entities in a knowledge graph.
We optimize the model to cluster entities of similar type by placing them inside the boxes corresponding to their relations.
arXiv Detail & Related papers (2023-05-19T22:42:16Z) - Prototype-based Embedding Network for Scene Graph Generation [105.97836135784794]
Current Scene Graph Generation (SGG) methods explore contextual information to predict relationships among entity pairs.
Due to the diverse visual appearance of numerous possible subject-object combinations, there is a large intra-class variation within each predicate category.
Prototype-based Embedding Network (PE-Net) models entities/predicates with prototype-aligned compact and distinctive representations.
PL is introduced to help PE-Net efficiently learn such entitypredicate matching, and Prototype Regularization (PR) is devised to relieve the ambiguous entity-predicate matching.
arXiv Detail & Related papers (2023-03-13T13:30:59Z) - Type-enriched Hierarchical Contrastive Strategy for Fine-Grained Entity
Typing [8.885149784531807]
Fine-grained entity typing aims to deduce specific semantic types of the entity mentions in text.
Few works directly model the type differences, that is, let models know the extent that one type is different from others.
Our method can directly model the differences between hierarchical types and improve the ability to distinguish multi-grained similar types.
arXiv Detail & Related papers (2022-08-22T06:38:08Z) - Language Model Cascades [72.18809575261498]
Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities.
Cases with control flow and dynamic structure require techniques from probabilistic programming.
We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
arXiv Detail & Related papers (2022-07-21T07:35:18Z) - Low-Rank Constraints for Fast Inference in Structured Models [110.38427965904266]
This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models.
Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces.
arXiv Detail & Related papers (2022-01-08T00:47:50Z) - Interpretable Entity Representations through Large-Scale Typing [61.4277527871572]
We present an approach to creating entity representations that are human readable and achieve high performance out of the box.
Our representations are vectors whose values correspond to posterior probabilities over fine-grained entity types.
We show that it is possible to reduce the size of our type set in a learning-based way for particular domains.
arXiv Detail & Related papers (2020-04-30T23:58:03Z) - E2EET: From Pipeline to End-to-end Entity Typing via Transformer-Based
Embeddings [7.431445082017672]
We propose a new type of entity typing called Entity Typing (ET)
ET involves labelling each entity mention with one or more class labels.
We propose to incorporate context using transformer-based embeddings for a mention-level model, and an end-to-end model using a Bi-GRU.
An extensive ablative study demonstrates the effectiveness of contextualised embeddings for mention-level models and the competitiveness of our end-to-end model for entity typing.
arXiv Detail & Related papers (2020-03-23T06:46:28Z) - Improving Entity Linking by Modeling Latent Entity Type Information [25.33342677359822]
We propose to inject latent entity type information into the entity embeddings based on pre-trained BERT.
In addition, we integrate a BERT-based entity similarity score into the local context model of a state-of-the-art model to better capture latent entity type information.
Our model significantly outperforms the state-of-the-art entity linking models on standard benchmark (AIDA-CoNLL)
arXiv Detail & Related papers (2020-01-06T09:18:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.