Convolutional Hypercomplex Embeddings for Link Prediction

• URL: http://arxiv.org/abs/2106.15230v1
• Date: Tue, 29 Jun 2021 10:26:51 GMT
• Title: Convolutional Hypercomplex Embeddings for Link Prediction
• Authors: Caglar Demir, Diego Moussallem, Stefan Heindorf, Axel-Cyrille Ngonga Ngomo
• Abstract summary: We propose QMult, OMult, ConvQ and ConvO to tackle the link prediction problem. QMult, OMult, ConvQ and ConvO build upon QMult and OMult by including convolution operations in a way inspired by the residual learning framework. We evaluate our approaches on seven link prediction datasets including WN18RR, FB15K-237 and YAGO3-10.
• Score: 2.6209112069534046
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Knowledge graph embedding research has mainly focused on the two smallest normed division algebras, $\mathbb{R}$ and $\mathbb{C}$. Recent results suggest that trilinear products of quaternion-valued embeddings can be a more effective means to tackle link prediction. In addition, models based on convolutions on real-valued embeddings often yield state-of-the-art results for link prediction. In this paper, we investigate a composition of convolution operations with hypercomplex multiplications. We propose the four approaches QMult, OMult, ConvQ and ConvO to tackle the link prediction problem. QMult and OMult can be considered as quaternion and octonion extensions of previous state-of-the-art approaches, including DistMult and ComplEx. ConvQ and ConvO build upon QMult and OMult by including convolution operations in a way inspired by the residual learning framework. We evaluated our approaches on seven link prediction datasets including WN18RR, FB15K-237 and YAGO3-10. Experimental results suggest that the benefits of learning hypercomplex-valued vector representations become more apparent as the size and complexity of the knowledge graph grows. ConvO outperforms state-of-the-art approaches on FB15K-237 in MRR, Hit@1 and Hit@3, while QMult, OMult, ConvQ and ConvO outperform state-of-the-approaches on YAGO3-10 in all metrics. Results also suggest that link prediction performances can be further improved via prediction averaging. To foster reproducible research, we provide an open-source implementation of approaches, including training and evaluation scripts as well as pretrained models.

Related papers

• Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regression [53.15502562048627]
  Recent work has built the connection between self-supervised learning and the approximation of the top eigenspace of a graph Laplacian operator. This work delves into a statistical analysis of augmentation-based pretraining.
  arXiv  Detail & Related papers  (2023-06-01T15:18:55Z)
• Quaternion-valued Correlation Learning for Few-Shot Semantic Segmentation [33.88445464404075]
  Few-shot segmentation (FSS) aims to segment unseen classes given only a few samples. We introduce a quaternion perspective on correlation learning and propose a novel Quaternion-valued Correlation Learning Network (QCLNet) Our QCLNet is formulated as a hyper-complex valued network and represents correlation tensors in the quaternion domain, which uses quaternion-valued convolution to explore the external relations of query subspace.
  arXiv  Detail & Related papers  (2023-05-12T06:56:22Z)
• Normalizing Flow-based Neural Process for Few-Shot Knowledge Graph Completion [69.55700751102376]
  Few-shot knowledge graph completion (FKGC) aims to predict missing facts for unseen relations with few-shot associated facts. Existing FKGC methods are based on metric learning or meta-learning, which often suffer from the out-of-distribution and overfitting problems. In this paper, we propose a normalizing flow-based neural process for few-shot knowledge graph completion (NP-FKGC)
  arXiv  Detail & Related papers  (2023-04-17T11:42:28Z)
• Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
  Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
  arXiv  Detail & Related papers  (2021-06-06T19:08:53Z)
• Out-of-Vocabulary Entities in Link Prediction [1.9036571490366496]
  Link prediction is often used as a proxy to evaluate the quality of embeddings. As benchmarks are crucial for the fair comparison of algorithms, ensuring their quality is tantamount to providing a solid ground for developing better solutions. We provide an implementation of an approach for spotting and removing such entities and provide corrected versions of the datasets WN18RR, FB15K-237, and YAGO3-10.
  arXiv  Detail & Related papers  (2021-05-26T12:58:18Z)
• Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling? [59.820507600960745]
  We propose a new GCP meta-layer that uses SVD in the forward pass, and Pad'e Approximants in the backward propagation to compute the gradients. The proposed meta-layer has been integrated into different CNN models and achieves state-of-the-art performances on both large-scale and fine-grained datasets.
  arXiv  Detail & Related papers  (2021-05-06T08:03:45Z)
• A shallow neural model for relation prediction [2.2559617939136505]
  We propose a shallow neural model (SHALLOM) that accurately infers missing relations from entities. Our experiments indicate that SHALLOM outperforms state-of-the-art approaches on the FB15K-237 and WN18RR datasets.
  arXiv  Detail & Related papers  (2021-01-22T13:10:11Z)
• Knowledge Generation -- Variational Bayes on Knowledge Graphs [0.685316573653194]
  This thesis is a proof of concept for potential of Vari Auto-Encoder (VAE) on representation of real-world Knowledge Graphs. Inspired by successful approaches to generation graphs, we evaluate the capabilities of our model, the Variational Auto-Encoder (RGVAE) The RGVAE is first evaluated on link prediction. The mean reciprocal rank (MRR) scores on the two FB15K-237 and WN18RR datasets are compared. We investigate the latent space in a twofold experiment: first, linear between the latent representation of two triples, then the exploration of each
  arXiv  Detail & Related papers  (2021-01-21T21:23:17Z)
• RatE: Relation-Adaptive Translating Embedding for Knowledge Graph Completion [51.64061146389754]
  We propose a relation-adaptive translation function built upon a novel weighted product in complex space. We then present our Relation-adaptive translating Embedding (RatE) approach to score each graph triple.
  arXiv  Detail & Related papers  (2020-10-10T01:30:30Z)
• Convolutional Complex Knowledge Graph Embeddings [1.1650381752104297]
  We present a new approach called ConEx, which infers missing links by leveraging a 2D convolution with a Hermitian inner product of complex-valued embedding vectors. We evaluate ConEx against state-of-the-art approaches on the WN18RR, FB15K-237, KINSHIP and UMLS benchmark datasets.
  arXiv  Detail & Related papers  (2020-08-07T12:49:01Z)
• Heuristic Semi-Supervised Learning for Graph Generation Inspired by Electoral College [80.67842220664231]
  We propose a novel pre-processing technique, namely ELectoral COllege (ELCO), which automatically expands new nodes and edges to refine the label similarity within a dense subgraph. In all setups tested, our method boosts the average score of base models by a large margin of 4.7 points, as well as consistently outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2020-06-10T14:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.