ConvD: Attention Enhanced Dynamic Convolutional Embeddings for Knowledge Graph Completion

• URL: http://arxiv.org/abs/2312.07589v1
• Date: Mon, 11 Dec 2023 07:37:58 GMT
• Title: ConvD: Attention Enhanced Dynamic Convolutional Embeddings for Knowledge Graph Completion
• Authors: Wenbin Guo, Zhao Li, Xin Wang, Zirui Chen
• Abstract summary: We propose a novel dynamic convolutional embedding model ConvD for knowledge graph completion. Our proposed model consistently outperforms the state-of-the-art baseline methods.
• Score: 11.223893397502431
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge graphs generally suffer from incompleteness, which can be alleviated by completing the missing information. Deep knowledge convolutional embedding models based on neural networks are currently popular methods for knowledge graph completion. However, most existing methods use external convolution kernels and traditional plain convolution processes, which limits the feature interaction capability of the model. In this paper, we propose a novel dynamic convolutional embedding model ConvD for knowledge graph completion, which directly reshapes the relation embeddings into multiple internal convolution kernels to improve the external convolution kernels of the traditional convolutional embedding model. The internal convolution kernels can effectively augment the feature interaction between the relation embeddings and entity embeddings, thus enhancing the model embedding performance. Moreover, we design a priori knowledge-optimized attention mechanism, which can assign different contribution weight coefficients to multiple relation convolution kernels for dynamic convolution to improve the expressiveness of the model further. Extensive experiments on various datasets show that our proposed model consistently outperforms the state-of-the-art baseline methods, with average improvements ranging from 11.30\% to 16.92\% across all model evaluation metrics. Ablation experiments verify the effectiveness of each component module of the ConvD model.

Related papers

• A Collaborative Ensemble Framework for CTR Prediction [73.59868761656317]
  We propose a novel framework, Collaborative Ensemble Training Network (CETNet), to leverage multiple distinct models. Unlike naive model scaling, our approach emphasizes diversity and collaboration through collaborative learning. We validate our framework on three public datasets and a large-scale industrial dataset from Meta.
  arXiv  Detail & Related papers  (2024-11-20T20:38:56Z)
• Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling [2.1779479916071067]
  We introduce a novel framework that enhances diffusion models by supporting a broader range of forward processes. We also propose a novel parameterization technique for learning the forward process. Results underscore NFDM's versatility and its potential for a wide range of applications.
  arXiv  Detail & Related papers  (2024-04-19T15:10:54Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Generative Learning of Continuous Data by Tensor Networks [45.49160369119449]
  We introduce a new family of tensor network generative models for continuous data. We benchmark the performance of this model on several synthetic and real-world datasets. Our methods give important theoretical and empirical evidence of the efficacy of quantum-inspired methods for the rapidly growing field of generative learning.
  arXiv  Detail & Related papers  (2023-10-31T14:37:37Z)
• Dynamic Tensor Decomposition via Neural Diffusion-Reaction Processes [24.723536390322582]
  tensor decomposition is an important tool for multiway data analysis. We propose Dynamic EMbedIngs fOr dynamic algorithm dEcomposition (DEMOTE) We show the advantage of our approach in both simulation study and real-world applications.
  arXiv  Detail & Related papers  (2023-10-30T15:49:45Z)
• Dynamic Kernel-Based Adaptive Spatial Aggregation for Learned Image Compression [63.56922682378755]
  We focus on extending spatial aggregation capability and propose a dynamic kernel-based transform coding. The proposed adaptive aggregation generates kernel offsets to capture valid information in the content-conditioned range to help transform. Experimental results demonstrate that our method achieves superior rate-distortion performance on three benchmarks compared to the state-of-the-art learning-based methods.
  arXiv  Detail & Related papers  (2023-08-17T01:34:51Z)
• ProjB: An Improved Bilinear Biased ProjE model for Knowledge Graph Completion [1.5576879053213302]
  This work improves on ProjE KGE due to low computational complexity and high potential for model improvement. Experimental results on benchmark Knowledge Graphs (KGs) such as FB15K and WN18 show that the proposed approach outperforms the state-of-the-art models in entity prediction task.
  arXiv  Detail & Related papers  (2022-08-15T18:18:05Z)
• Data-heterogeneity-aware Mixing for Decentralized Learning [63.83913592085953]
  We characterize the dependence of convergence on the relationship between the mixing weights of the graph and the data heterogeneity across nodes. We propose a metric that quantifies the ability of a graph to mix the current gradients. Motivated by our analysis, we propose an approach that periodically and efficiently optimize the metric.
  arXiv  Detail & Related papers  (2022-04-13T15:54:35Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Sparse Flows: Pruning Continuous-depth Models [107.98191032466544]
  We show that pruning improves generalization for neural ODEs in generative modeling. We also show that pruning finds minimal and efficient neural ODE representations with up to 98% less parameters compared to the original network, without loss of accuracy.
  arXiv  Detail & Related papers  (2021-06-24T01:40:17Z)
• Kernel-Based Models for Influence Maximization on Graphs based on Gaussian Process Variance Minimization [9.357483974291899]
  We introduce and investigate a novel model for influence (IM) on graphs. Data-driven approaches can be applied to determine proper kernels for this IM model. Compared to models in this field that rely on costly Monte-Carlo simulations, our model allows for a simple and cost-efficient update strategy.
  arXiv  Detail & Related papers  (2021-03-02T08:55:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.