Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding

• URL: http://arxiv.org/abs/2302.01849v1
• Date: Fri, 3 Feb 2023 16:49:46 GMT
• Title: Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding
• Authors: Mingyang Chen, Wen Zhang, Zhen Yao, Yushan Zhu, Yang Gao, Jeff Z. Pan, Huajun Chen
• Abstract summary: We propose an entity-agnostic representation learning method for handling the problem of inefficient parameter storage costs brought by embedding knowledge graphs. We learn universal and entity-agnostic encoders for transforming distinguishable information into entity embeddings. Experimental results show that EARL uses fewer parameters and performs better on link prediction tasks than baselines.
• Score: 30.7075844882004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose an entity-agnostic representation learning method for handling the problem of inefficient parameter storage costs brought by embedding knowledge graphs. Conventional knowledge graph embedding methods map elements in a knowledge graph, including entities and relations, into continuous vector spaces by assigning them one or multiple specific embeddings (i.e., vector representations). Thus the number of embedding parameters increases linearly as the growth of knowledge graphs. In our proposed model, Entity-Agnostic Representation Learning (EARL), we only learn the embeddings for a small set of entities and refer to them as reserved entities. To obtain the embeddings for the full set of entities, we encode their distinguishable information from their connected relations, k-nearest reserved entities, and multi-hop neighbors. We learn universal and entity-agnostic encoders for transforming distinguishable information into entity embeddings. This approach allows our proposed EARL to have a static, efficient, and lower parameter count than conventional knowledge graph embedding methods. Experimental results show that EARL uses fewer parameters and performs better on link prediction tasks than baselines, reflecting its parameter efficiency.

Related papers

• Inference over Unseen Entities, Relations and Literals on Knowledge Graphs [1.7474352892977463]
  knowledge graph embedding models have been successfully applied in the transductive setting to tackle various challenging tasks. We propose the attentive byte-pair encoding layer (BytE) to construct a triple embedding from a sequence of byte-pair encoded subword units of entities and relations. BytE leads to massive feature reuse via weight tying, since it forces a knowledge graph embedding model to learn embeddings for subword units instead of entities and relations directly.
  arXiv  Detail & Related papers  (2024-10-09T10:20:54Z)
• Knowledge Graph Structure as Prompt: Improving Small Language Models Capabilities for Knowledge-based Causal Discovery [10.573861741540853]
  KG Structure as Prompt is a novel approach for integrating structural information from a knowledge graph, such as common neighbor nodes and metapaths, into prompt-based learning. Experimental results on three types of biomedical and open-domain datasets under few-shot settings demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2024-07-26T14:07:00Z)
• Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective [106.92016199403042]
  We empirically investigate knowledge transfer from larger to smaller models through a parametric perspective. We employ sensitivity-based techniques to extract and align knowledge-specific parameters between different large language models. Our findings highlight the critical factors contributing to the process of parametric knowledge transfer.
  arXiv  Detail & Related papers  (2023-10-17T17:58:34Z)
• Learning Representations without Compositional Assumptions [79.12273403390311]
  We propose a data-driven approach that learns feature set dependencies by representing feature sets as graph nodes and their relationships as learnable edges. We also introduce LEGATO, a novel hierarchical graph autoencoder that learns a smaller, latent graph to aggregate information from multiple views dynamically.
  arXiv  Detail & Related papers  (2023-05-31T10:36:10Z)
• Schema-aware Reference as Prompt Improves Data-Efficient Knowledge Graph Construction [57.854498238624366]
  We propose a retrieval-augmented approach, which retrieves schema-aware Reference As Prompt (RAP) for data-efficient knowledge graph construction. RAP can dynamically leverage schema and knowledge inherited from human-annotated and weak-supervised data as a prompt for each sample.
  arXiv  Detail & Related papers  (2022-10-19T16:40:28Z)
• Virtual Relational Knowledge Graphs for Recommendation [15.978408290522852]
  We argue that it is not efficient nor effective to use every relation type for item encoding. We first construct virtual relational graphs (VRKGs) by an unsupervised learning scheme. We also employ the LWS mechanism on a user-item bipartite graph for user representation learning.
  arXiv  Detail & Related papers  (2022-04-03T15:14:20Z)
• Reasoning Through Memorization: Nearest Neighbor Knowledge Graph Embeddings [29.94706167233985]
  kNN-KGE is a new knowledge graph embedding approach with pre-trained language models. We compute the nearest neighbors based on the distance in the entity embedding space from the knowledge store.
  arXiv  Detail & Related papers  (2022-01-14T17:35:16Z)
• Why Settle for Just One? Extending EL++ Ontology Embeddings with Many-to-Many Relationships [2.599882743586164]
  Knowledge Graph embeddings provide a low-dimensional representation of entities and relations of a Knowledge Graph. Recent efforts in this direction involve learning embeddings for a Description (logical Logic for a description) named EL++. We provide a simple and effective solution that allows such methods to consider many-to-many relationships while learning embedding representations.
  arXiv  Detail & Related papers  (2021-10-20T13:23:18Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)
• Relational Message Passing for Knowledge Graph Completion [78.47976646383222]
  We propose a relational message passing method for knowledge graph completion. It passes relational messages among edges iteratively to aggregate neighborhood information. Results show our method outperforms stateof-the-art knowledge completion methods by a large margin.
  arXiv  Detail & Related papers  (2020-02-17T03:33:41Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.