ModulE: Module Embedding for Knowledge Graphs

• URL: http://arxiv.org/abs/2203.04702v1
• Date: Wed, 9 Mar 2022 13:27:46 GMT
• Title: ModulE: Module Embedding for Knowledge Graphs
• Authors: Jingxuan Chai and Guangming Shi
• Abstract summary: Knowledge graph embedding (KGE) has been shown to be a powerful tool for predicting missing links of a knowledge graph. We propose a novel group theory-based embedding framework for rotation-based models, in which both entities and relations are embedded as group elements. Specifically, under our framework, we introduce a more generic embedding method, ModulE, which projects entities to a module.
• Score: 33.603174694114095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Knowledge graph embedding (KGE) has been shown to be a powerful tool for predicting missing links of a knowledge graph. However, existing methods mainly focus on modeling relation patterns, while simply embed entities to vector spaces, such as real field, complex field and quaternion space. To model the embedding space from a more rigorous and theoretical perspective, we propose a novel general group theory-based embedding framework for rotation-based models, in which both entities and relations are embedded as group elements. Furthermore, in order to explore more available KGE models, we utilize a more generic group structure, module, a generalization notion of vector space. Specifically, under our framework, we introduce a more generic embedding method, ModulE, which projects entities to a module. Following the method of ModulE, we build three instantiating models: ModulE$_{\mathbb{R},\mathbb{C}}$, ModulE$_{\mathbb{R},\mathbb{H}}$ and ModulE$_{\mathbb{H},\mathbb{H}}$, by adopting different module structures. Experimental results show that ModulE$_{\mathbb{H},\mathbb{H}}$ which embeds entities to a module over non-commutative ring, achieves state-of-the-art performance on multiple benchmark datasets.

Related papers

• GeoMFormer: A General Architecture for Geometric Molecular Representation Learning [84.02083170392764]
  We introduce a novel Transformer-based molecular model called GeoMFormer to achieve this goal. We show that GeoMFormer achieves strong performance on both invariant and equivariant tasks of different types and scales.
  arXiv  Detail & Related papers  (2024-06-24T17:58:13Z)
• ModuleFormer: Modularity Emerges from Mixture-of-Experts [60.6148988099284]
  This paper proposes a new neural network architecture, ModuleFormer, to improve the efficiency and flexibility of large language models. Unlike the previous SMoE-based modular language model, ModuleFormer can induce modularity from uncurated data.
  arXiv  Detail & Related papers  (2023-06-07T17:59:57Z)
• Modular Deep Learning [120.36599591042908]
  Transfer learning has recently become the dominant paradigm of machine learning. It remains unclear how to develop models that specialise towards multiple tasks without incurring negative interference. Modular deep learning has emerged as a promising solution to these challenges.
  arXiv  Detail & Related papers  (2023-02-22T18:11:25Z)
• From Torus Bundles to Particle-Hole Equivariantization [15.857538570676667]
  We consider an infinite family of 3-manifolds, that is, torus bundles over the circle. We show that the modular data are realized by the $mathbbZ$-equivariantization of certain pointed premodular categories. It is our hope that this extensive class of examples will shed light on how to improve the program to recover the full data of a premodular category.
  arXiv  Detail & Related papers  (2021-06-03T16:06:26Z)
• K\"ahler Geometry of Quiver Varieties and Machine Learning [0.0]
  We develop an algebro-geometric formulation for neural networks in machine learning using the moduli space of framed representations quiver. We prove the universal approximation theorem for the multi-variable activation function constructed from the complex projective space.
  arXiv  Detail & Related papers  (2021-01-27T15:32:24Z)
• From Three Dimensional Manifolds to Modular Tensor Categories [0.7877961820015922]
  Cho, Gang, and Kim outlined a program that connects topology and quantum topology. It is conjectured that every modular tensor category can be obtained from a three manifold and a Lie group. We study this program mathematically, and provide strong support for the feasibility of such a program.
  arXiv  Detail & Related papers  (2021-01-05T17:45:23Z)
• AutoETER: Automated Entity Type Representation for Knowledge Graph Embedding [40.900070190077024]
  We develop a novel Knowledge Graph Embedding (KGE) framework with Automated Entity TypE Representation (AutoETER) Our approach could model and infer all the relation patterns and complex relations. Experiments on four datasets demonstrate the superior performance of our model compared to state-of-the-art baselines on link prediction tasks.
  arXiv  Detail & Related papers  (2020-09-25T04:27:35Z)
• S2RMs: Spatially Structured Recurrent Modules [105.0377129434636]
  We take a step towards exploiting dynamic structure that are capable of simultaneously exploiting both modular andtemporal structures. We find our models to be robust to the number of available views and better capable of generalization to novel tasks without additional training.
  arXiv  Detail & Related papers  (2020-07-13T17:44:30Z)
• NagE: Non-Abelian Group Embedding for Knowledge Graphs [14.545770519120898]
  We show that a group-based embedding framework is essential for designing embedding models. Motivated by theoretical analysis, we have proposed a group theory-based knowledge graph embedding framework. We provide a generic recipe to construct embedding models associated with two instantiating examples.
  arXiv  Detail & Related papers  (2020-05-22T00:54:45Z)
• Bipartite Flat-Graph Network for Nested Named Entity Recognition [94.91507634620133]
  Bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER) We propose a novel bipartite flat-graph network (BiFlaG) for nested named entity recognition (NER)
  arXiv  Detail & Related papers  (2020-05-01T15:14:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.