Extending TWIG: Zero-Shot Predictive Hyperparameter Selection for KGEs based on Graph Structure

• URL: http://arxiv.org/abs/2412.14801v1
• Date: Thu, 19 Dec 2024 12:47:21 GMT
• Title: Extending TWIG: Zero-Shot Predictive Hyperparameter Selection for KGEs based on Graph Structure
• Authors: Jeffrey Sardina, John D. Kelleher, Declan O'Sullivan,
• Abstract summary: The Topologically-Weighted Intelligence Generation (TWIG) model has been proposed as a solution to modelling how each of these elements relate. We extend the previous research on TWIG and evaluate its ability to simulate the output of the KGE model ComplEx in the cross-KG setting.
• Score: 2.2690868277262486
• License:
• Abstract: Knowledge Graphs (KGs) have seen increasing use across various domains -- from biomedicine and linguistics to general knowledge modelling. In order to facilitate the analysis of knowledge graphs, Knowledge Graph Embeddings (KGEs) have been developed to automatically analyse KGs and predict new facts based on the information in a KG, a task called "link prediction". Many existing studies have documented that the structure of a KG, KGE model components, and KGE hyperparameters can significantly change how well KGEs perform and what relationships they are able to learn. Recently, the Topologically-Weighted Intelligence Generation (TWIG) model has been proposed as a solution to modelling how each of these elements relate. In this work, we extend the previous research on TWIG and evaluate its ability to simulate the output of the KGE model ComplEx in the cross-KG setting. Our results are twofold. First, TWIG is able to summarise KGE performance on a wide range of hyperparameter settings and KGs being learned, suggesting that it represents a general knowledge of how to predict KGE performance from KG structure. Second, we show that TWIG can successfully predict hyperparameter performance on unseen KGs in the zero-shot setting. This second observation leads us to propose that, with additional research, optimal hyperparameter selection for KGE models could be determined in a pre-hoc manner using TWIG-like methods, rather than by using a full hyperparameter search.

Related papers

• TWIG: Towards pre-hoc Hyperparameter Optimisation and Cross-Graph Generalisation via Simulated KGE Models [2.550226198121927]
  We introduce TWIG (Topologically-Weighted Intelligence Generation), a novel, embedding-free paradigm for simulating the output of KGEs. Experiments on the UMLS dataset show that a single TWIG neural network can predict the results of state-of-the-art ComplEx-N3 KGE model.
  arXiv  Detail & Related papers  (2024-02-08T23:12:02Z)
• A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning [49.09125100268454]
  Knowledge Graph Embedding (KGE) has proven to be an effective approach to solving the Knowledge Completion Graph (KGC) task. Relational patterns are an important factor in the performance of KGE models. We introduce a training-free method to enhance KGE models' performance over various relational patterns.
  arXiv  Detail & Related papers  (2023-08-15T17:30:57Z)
• KGxBoard: Explainable and Interactive Leaderboard for Evaluation of Knowledge Graph Completion Models [76.01814380927507]
  KGxBoard is an interactive framework for performing fine-grained evaluation on meaningful subsets of the data. In our experiments, we highlight the findings with the use of KGxBoard, which would have been impossible to detect with standard averaged single-score metrics.
  arXiv  Detail & Related papers  (2022-08-23T15:11:45Z)
• Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network [111.67744771462873]
  This paper proposes a novel explainable model for sparse Knowledge Graphs (KGs) It combines high-order reasoning into a graph convolutional network, namely HoGRN. It can not only improve the generalization ability to mitigate the information insufficiency issue but also provide interpretability.
  arXiv  Detail & Related papers  (2022-07-14T10:16:56Z)
• Start Small, Think Big: On Hyperparameter Optimization for Large-Scale Knowledge Graph Embeddings [4.3400407844815]
  We introduce an efficient multi-fidelity HPO algorithm for large-scale knowledge graphs. GraSH obtains state-of-the-art results on large graphs at a low cost.
  arXiv  Detail & Related papers  (2022-07-11T16:07:16Z)
• ExpressivE: A Spatio-Functional Embedding For Knowledge Graph Completion [78.8942067357231]
  ExpressivE embeds pairs of entities as points and relations as hyper-parallelograms in the virtual triple space. We show that ExpressivE is competitive with state-of-the-art KGEs and even significantly outperforms them on W18RR.
  arXiv  Detail & Related papers  (2022-06-08T23:34:39Z)
• KQGC: Knowledge Graph Embedding with Smoothing Effects of Graph Convolutions for Recommendation [3.264007084815591]
  We propose a new model for recommender systems named Knowledge Query-based Graph Convolution (KQGC) KQGC focuses on the smoothing, and leverages a simple linear graph convolution for smoothing KGE. We apply the proposed KQGC to a recommendation task that aims prospective users for specific products.
  arXiv  Detail & Related papers  (2022-05-23T09:34:06Z)
• Sequence-to-Sequence Knowledge Graph Completion and Question Answering [8.207403859762044]
  We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding.
  arXiv  Detail & Related papers  (2022-03-19T13:01:49Z)
• Rethinking Graph Convolutional Networks in Knowledge Graph Completion [83.25075514036183]
  Graph convolutional networks (GCNs) have been increasingly popular in knowledge graph completion (KGC) In this paper, we build upon representative GCN-based KGC models and introduce variants to find which factor of GCNs is critical in KGC. We propose a simple yet effective framework named LTE-KGE, which equips existing KGE models with linearly transformed entity embeddings.
  arXiv  Detail & Related papers  (2022-02-08T11:36:18Z)
• How Does Knowledge Graph Embedding Extrapolate to Unseen Data: a Semantic Evidence View [13.575052133743505]
  We study how does Knowledge Graph Embedding (KGE) extrapolate to unseen data. We also propose a novel GNN-based KGE model, called Semantic Evidence aware Graph Neural Network (SE-GNN)
  arXiv  Detail & Related papers  (2021-09-24T08:17:02Z)
• RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding [50.010601631982425]
  This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) We derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail) We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph.
  arXiv  Detail & Related papers  (2021-01-25T13:31:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.