Link Prediction with Attention Applied on Multiple Knowledge Graph Embedding Models

• URL: http://arxiv.org/abs/2302.06229v1
• Date: Mon, 13 Feb 2023 10:07:26 GMT
• Title: Link Prediction with Attention Applied on Multiple Knowledge Graph Embedding Models
• Authors: Cosimo Gregucci and Mojtaba Nayyeri and Daniel Hern\'andez and Steffen Staab
• Abstract summary: Knowledge graph embeddings map nodes into a vector space to predict new links, scoring them according to geometric criteria. No single model can learn all patterns equally well. In this paper, we combine the query representations from several models in a unified one to incorporate patterns that are independently captured by each model.
• Score: 7.620967781722715
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Predicting missing links between entities in a knowledge graph is a fundamental task to deal with the incompleteness of data on the Web. Knowledge graph embeddings map nodes into a vector space to predict new links, scoring them according to geometric criteria. Relations in the graph may follow patterns that can be learned, e.g., some relations might be symmetric and others might be hierarchical. However, the learning capability of different embedding models varies for each pattern and, so far, no single model can learn all patterns equally well. In this paper, we combine the query representations from several models in a unified one to incorporate patterns that are independently captured by each model. Our combination uses attention to select the most suitable model to answer each query. The models are also mapped onto a non-Euclidean manifold, the Poincar\'e ball, to capture structural patterns, such as hierarchies, besides relational patterns, such as symmetry. We prove that our combination provides a higher expressiveness and inference power than each model on its own. As a result, the combined model can learn relational and structural patterns. We conduct extensive experimental analysis with various link prediction benchmarks showing that the combined model outperforms individual models, including state-of-the-art approaches.

Related papers

• Knowledge Graph Embeddings: A Comprehensive Survey on Capturing Relation Properties [5.651919225343915]
  Knowledge Graph Embedding (KGE) techniques play a pivotal role in transforming symbolic Knowledge Graphs into numerical representations. This paper addresses the complex mapping properties inherent in relations, such as one-to-one, one-to-many, many-to-one, and many-to-many mappings. We explore innovative ideas such as integrating multimodal information into KGE, enhancing relation pattern modeling with rules, and developing models to capture relation characteristics in dynamic KGE settings.
  arXiv  Detail & Related papers  (2024-10-16T08:54:52Z)
• Relational Learning in Pre-Trained Models: A Theory from Hypergraph Recovery Perspective [60.64922606733441]
  We introduce a mathematical model that formalizes relational learning as hypergraph recovery to study pre-training of Foundation Models (FMs) In our framework, the world is represented as a hypergraph, with data abstracted as random samples from hyperedges. We theoretically examine the feasibility of a Pre-Trained Model (PTM) to recover this hypergraph and analyze the data efficiency in a minimax near-optimal style.
  arXiv  Detail & Related papers  (2024-06-17T06:20:39Z)
• A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning [49.09125100268454]
  Knowledge Graph Embedding (KGE) has proven to be an effective approach to solving the Knowledge Completion Graph (KGC) task. Relational patterns are an important factor in the performance of KGE models. We introduce a training-free method to enhance KGE models' performance over various relational patterns.
  arXiv  Detail & Related papers  (2023-08-15T17:30:57Z)
• G-MSM: Unsupervised Multi-Shape Matching with Graph-based Affinity Priors [52.646396621449]
  G-MSM is a novel unsupervised learning approach for non-rigid shape correspondence. We construct an affinity graph on a given set of training shapes in a self-supervised manner. We demonstrate state-of-the-art performance on several recent shape correspondence benchmarks.
  arXiv  Detail & Related papers  (2022-12-06T12:09:24Z)
• GrannGAN: Graph annotation generative adversarial networks [72.66289932625742]
  We consider the problem of modelling high-dimensional distributions and generating new examples of data with complex relational feature structure coherent with a graph skeleton. The model we propose tackles the problem of generating the data features constrained by the specific graph structure of each data point by splitting the task into two phases. In the first it models the distribution of features associated with the nodes of the given graph, in the second it complements the edge features conditionally on the node features.
  arXiv  Detail & Related papers  (2022-12-01T11:49:07Z)
• Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones [64.75766944882389]
  We present ConE (Cone Embedding), a KG embedding model that is able to simultaneously model multiple hierarchical as well as non-hierarchical relations in a knowledge graph. In particular, ConE uses cone containment constraints in different subspaces of the hyperbolic embedding space to capture multiple heterogeneous hierarchies. Our approach yields new state-of-the-art Hits@1 of 45.3% on WN18RR and 16.1% on DDB14 (0.231 MRR)
  arXiv  Detail & Related papers  (2021-10-28T07:16:08Z)
• An Ample Approach to Data and Modeling [1.0152838128195467]
  We describe a framework for modeling how models can be built that integrates concepts and methods from a wide range of fields. The reference M* meta model framework is presented, which relies critically in associating whole datasets and respective models in terms of a strict equivalence relation. Several considerations about how the developed framework can provide insights about data clustering, complexity, collaborative research, deep learning, and creativity are then presented.
  arXiv  Detail & Related papers  (2021-10-05T01:26:09Z)
• Is There More Pattern in Knowledge Graph? Exploring Proximity Pattern for Knowledge Graph Embedding [13.17623081024394]
  We name such semantic phenomenon in knowledge graph as proximity pattern. With the original knowledge graph, we design a Chained couPle-GNN architecture to deeply merge the two patterns. Being evaluated on FB15k-237 and WN18RR datasets, CP-GNN achieves state-of-the-art results for Knowledge Graph Completion task.
  arXiv  Detail & Related papers  (2021-10-02T03:50:42Z)
• New Ideas for Brain Modelling 7 [0.0]
  This paper updates the cognitive model by creating two systems and unifying them over the same structure. It represents information at the semantic level only, where labelled patterns are aggregated into a 'type-set-match' form.
  arXiv  Detail & Related papers  (2020-11-04T10:59:01Z)
• CoSE: Compositional Stroke Embeddings [52.529172734044664]
  We present a generative model for complex free-form structures such as stroke-based drawing tasks. Our approach is suitable for interactive use cases such as auto-completing diagrams.
  arXiv  Detail & Related papers  (2020-06-17T15:22:54Z)
• LineaRE: Simple but Powerful Knowledge Graph Embedding for Link Prediction [7.0294164380111015]
  We propose a novel embedding model, namely LineaRE, which is capable of modeling four connectivity patterns and four mapping properties. Experimental results on multiple widely used real-world datasets show that the proposed LineaRE model significantly outperforms existing state-of-the-art models for link prediction tasks.
  arXiv  Detail & Related papers  (2020-04-21T14:19:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.