Sharing Parameter by Conjugation for Knowledge Graph Embeddings in Complex Space

• URL: http://arxiv.org/abs/2404.11809v1
• Date: Thu, 18 Apr 2024 00:05:02 GMT
• Title: Sharing Parameter by Conjugation for Knowledge Graph Embeddings in Complex Space
• Authors: Xincan Feng, Zhi Qu, Yuchang Cheng, Taro Watanabe, Nobuhiro Yugami,
• Abstract summary: A Knowledge Graph (KG) is the directed graphical representation of entities and relations in the real world. The need to scale up and complete KG automatically yields Knowledge Graph Embedding (KGE) We propose a parameter-sharing method, i.e., using conjugate parameters for complex numbers employed in KGE models. Our method improves memory efficiency by 2x in relation embedding while achieving comparable performance to the state-of-the-art non-conjugate models.
• Score: 11.337803135227752
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A Knowledge Graph (KG) is the directed graphical representation of entities and relations in the real world. KG can be applied in diverse Natural Language Processing (NLP) tasks where knowledge is required. The need to scale up and complete KG automatically yields Knowledge Graph Embedding (KGE), a shallow machine learning model that is suffering from memory and training time consumption issues. To mitigate the computational load, we propose a parameter-sharing method, i.e., using conjugate parameters for complex numbers employed in KGE models. Our method improves memory efficiency by 2x in relation embedding while achieving comparable performance to the state-of-the-art non-conjugate models, with faster, or at least comparable, training time. We demonstrated the generalizability of our method on two best-performing KGE models $5^{\bigstar}\mathrm{E}$ and $\mathrm{ComplEx}$ on five benchmark datasets.

Related papers

• CL4KGE: A Curriculum Learning Method for Knowledge Graph Embedding [36.47838597326351]
  We define a metric Z-counts to measure the difficulty of training each triple in knowledge graphs. Based on this metric, we propose textbfCL4KGE, an efficient textbfCurriculum textbfLearning based training strategy.
  arXiv  Detail & Related papers  (2024-08-27T07:51:26Z)
• A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning [49.09125100268454]
  Knowledge Graph Embedding (KGE) has proven to be an effective approach to solving the Knowledge Completion Graph (KGC) task. Relational patterns are an important factor in the performance of KGE models. We introduce a training-free method to enhance KGE models' performance over various relational patterns.
  arXiv  Detail & Related papers  (2023-08-15T17:30:57Z)
• Schema First! Learn Versatile Knowledge Graph Embeddings by Capturing Semantics with MASCHInE [3.174882428337821]
  Knowledge graph embedding models (KGEMs) have gained considerable traction in recent years. In this work, we design protographs -- small, modified versions of a KG that leverage RDF/S information. The learnt protograph-based embeddings are meant to encapsulate the semantics of a KG, and can be leveraged in learning KGEs that, in turn, also better capture semantics.
  arXiv  Detail & Related papers  (2023-06-06T13:22:54Z)
• Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation [65.62538699160085]
  We propose a Directed Acyclic Graph Factorization Machine (KD-DAGFM) to learn the high-order feature interactions from existing complex interaction models for CTR prediction via Knowledge Distillation. KD-DAGFM achieves the best performance with less than 21.5% FLOPs of the state-of-the-art method on both online and offline experiments.
  arXiv  Detail & Related papers  (2022-11-21T03:09:42Z)
• GreenKGC: A Lightweight Knowledge Graph Completion Method [32.528770408502396]
  GreenKGC aims to discover missing relationships between entities in knowledge graphs. It consists of three modules: representation learning, feature pruning, and decision learning. In low dimensions, GreenKGC can outperform SOTA methods in most datasets.
  arXiv  Detail & Related papers  (2022-08-19T03:33:45Z)
• Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks [52.566735716983956]
  We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs. Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs. Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
  arXiv  Detail & Related papers  (2022-07-18T14:23:31Z)
• Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network [111.67744771462873]
  This paper proposes a novel explainable model for sparse Knowledge Graphs (KGs) It combines high-order reasoning into a graph convolutional network, namely HoGRN. It can not only improve the generalization ability to mitigate the information insufficiency issue but also provide interpretability.
  arXiv  Detail & Related papers  (2022-07-14T10:16:56Z)
• MEKER: Memory Efficient Knowledge Embedding Representation for Link Prediction and Question Answering [65.62309538202771]
  Knowledge Graphs (KGs) are symbolically structured storages of facts. KG embedding contains concise data used in NLP tasks requiring implicit information about the real world. We propose a memory-efficient KG embedding model, which yields SOTA-comparable performance on link prediction tasks and KG-based Question Answering.
  arXiv  Detail & Related papers  (2022-04-22T10:47:03Z)
• Sequence-to-Sequence Knowledge Graph Completion and Question Answering [8.207403859762044]
  We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding.
  arXiv  Detail & Related papers  (2022-03-19T13:01:49Z)
• A Simple But Powerful Graph Encoder for Temporal Knowledge Graph Completion [13.047205680129094]
  We propose a simple but powerful graph encoder TARGCN for temporal knowledge graphs (TKGs) Our model can achieve a more than 42% relative improvement on GDELT dataset compared with the state-of-the-art model. It outperforms the strongest baseline on ICEWS05-15 dataset with around 18.5% fewer parameters.
  arXiv  Detail & Related papers  (2021-12-14T23:30:42Z)
• Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction [123.20238648121445]
  We propose a new self-supervised learning framework, Graph Information Aided Node feature exTraction (GIANT) GIANT makes use of the eXtreme Multi-label Classification (XMC) formalism, which is crucial for fine-tuning the language model based on graph information. We demonstrate the superior performance of GIANT over the standard GNN pipeline on Open Graph Benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-29T19:55:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.