Inference over Unseen Entities, Relations and Literals on Knowledge Graphs

• URL: http://arxiv.org/abs/2410.06742v1
• Date: Wed, 9 Oct 2024 10:20:54 GMT
• Title: Inference over Unseen Entities, Relations and Literals on Knowledge Graphs
• Authors: Caglar Demir, N'Dah Jean Kouagou, Arnab Sharma, Axel-Cyrille Ngonga Ngomo,
• Abstract summary: knowledge graph embedding models have been successfully applied in the transductive setting to tackle various challenging tasks. We propose the attentive byte-pair encoding layer (BytE) to construct a triple embedding from a sequence of byte-pair encoded subword units of entities and relations. BytE leads to massive feature reuse via weight tying, since it forces a knowledge graph embedding model to learn embeddings for subword units instead of entities and relations directly.
• Score: 1.7474352892977463
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In recent years, knowledge graph embedding models have been successfully applied in the transductive setting to tackle various challenging tasks including link prediction, and query answering. Yet, the transductive setting does not allow for reasoning over unseen entities, relations, let alone numerical or non-numerical literals. Although increasing efforts are put into exploring inductive scenarios, inference over unseen entities, relations, and literals has yet to come. This limitation prohibits the existing methods from handling real-world dynamic knowledge graphs involving heterogeneous information about the world. Here, we propose a remedy to this limitation. We propose the attentive byte-pair encoding layer (BytE) to construct a triple embedding from a sequence of byte-pair encoded subword units of entities and relations. Compared to the conventional setting, BytE leads to massive feature reuse via weight tying, since it forces a knowledge graph embedding model to learn embeddings for subword units instead of entities and relations directly. Consequently, the size of the embedding matrices are not anymore bound to the unique number of entities and relations of a knowledge graph. Experimental results show that BytE improves the link prediction performance of 4 knowledge graph embedding models on datasets where the syntactic representations of triples are semantically meaningful. However, benefits of training a knowledge graph embedding model with BytE dissipate on knowledge graphs where entities and relations are represented with plain numbers or URIs. We provide an open source implementation of BytE to foster reproducible research.

Related papers

• InGram: Inductive Knowledge Graph Embedding via Relation Graphs [16.005051393690792]
  In this paper, we propose an INductive knowledge GRAph eMbedding method, InGram, that can generate embeddings of new relations as well as new entities at inference time. Experimental results show that InGram outperforms 14 different state-of-the-art methods on varied inductive learning scenarios.
  arXiv  Detail & Related papers  (2023-05-31T16:10:42Z)
• Learning Representations without Compositional Assumptions [79.12273403390311]
  We propose a data-driven approach that learns feature set dependencies by representing feature sets as graph nodes and their relationships as learnable edges. We also introduce LEGATO, a novel hierarchical graph autoencoder that learns a smaller, latent graph to aggregate information from multiple views dynamically.
  arXiv  Detail & Related papers  (2023-05-31T10:36:10Z)
• Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding [30.7075844882004]
  We propose an entity-agnostic representation learning method for handling the problem of inefficient parameter storage costs brought by embedding knowledge graphs. We learn universal and entity-agnostic encoders for transforming distinguishable information into entity embeddings. Experimental results show that EARL uses fewer parameters and performs better on link prediction tasks than baselines.
  arXiv  Detail & Related papers  (2023-02-03T16:49:46Z)
• RAILD: Towards Leveraging Relation Features for Inductive Link Prediction In Knowledge Graphs [1.5469452301122175]
  Relation Aware Inductive Link preDiction (RAILD) is proposed for Knowledge Graph completion. RAILD learns representations for both unseen entities and unseen relations.
  arXiv  Detail & Related papers  (2022-11-21T12:35:30Z)
• KGLM: Integrating Knowledge Graph Structure in Language Models for Link Prediction [0.0]
  We introduce a new entity/relation embedding layer that learns to differentiate distinctive entity and relation types. We show that further pre-training the language models with this additional embedding layer using the triples extracted from the knowledge graph, followed by the standard fine-tuning phase sets a new state-of-the-art performance for the link prediction task on the benchmark datasets.
  arXiv  Detail & Related papers  (2022-11-04T20:38:12Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Context-Enhanced Entity and Relation Embedding for Knowledge Graph Completion [2.580765958706854]
  We propose a model named AggrE, which conducts efficient aggregations on entity context and relation context in multi-hops. Experiment results show that AggrE is competitive to existing models.
  arXiv  Detail & Related papers  (2020-12-13T09:20:42Z)
• ENT-DESC: Entity Description Generation by Exploring Knowledge Graph [53.03778194567752]
  In practice, the input knowledge could be more than enough, since the output description may only cover the most significant knowledge. We introduce a large-scale and challenging dataset to facilitate the study of such a practical scenario in KG-to-text. We propose a multi-graph structure that is able to represent the original graph information more comprehensively.
  arXiv  Detail & Related papers  (2020-04-30T14:16:19Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)
• Relational Message Passing for Knowledge Graph Completion [78.47976646383222]
  We propose a relational message passing method for knowledge graph completion. It passes relational messages among edges iteratively to aggregate neighborhood information. Results show our method outperforms stateof-the-art knowledge completion methods by a large margin.
  arXiv  Detail & Related papers  (2020-02-17T03:33:41Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.