DensE: An Enhanced Non-commutative Representation for Knowledge Graph Embedding with Adaptive Semantic Hierarchy

• URL: http://arxiv.org/abs/2008.04548v2
• Date: Tue, 11 Jan 2022 04:18:08 GMT
• Title: DensE: An Enhanced Non-commutative Representation for Knowledge Graph Embedding with Adaptive Semantic Hierarchy
• Authors: Haonan Lu, Hailin Hu, Xiaodong Lin
• Abstract summary: We develop a novel knowledge graph embedding method, named DensE, to provide an improved modeling scheme for the complex composition patterns of relations. Our method decomposes each relation into an SO(3) group-based rotation operator and a scaling operator in the three dimensional (3-D) Euclidean space. Experimental results on multiple benchmark knowledge graphs show that DensE outperforms the current state-of-the-art models for missing link prediction.
• Score: 4.607120217372668
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Capturing the composition patterns of relations is a vital task in knowledge graph completion. It also serves as a fundamental step towards multi-hop reasoning over learned knowledge. Previously, several rotation-based translational methods have been developed to model composite relations using the product of a series of complex-valued diagonal matrices. However, these methods tend to make several oversimplified assumptions on the composite relations, e.g., forcing them to be commutative, independent from entities and lacking semantic hierarchy. To systematically tackle these problems, we have developed a novel knowledge graph embedding method, named DensE, to provide an improved modeling scheme for the complex composition patterns of relations. In particular, our method decomposes each relation into an SO(3) group-based rotation operator and a scaling operator in the three dimensional (3-D) Euclidean space. This design principle leads to several advantages of our method: (1) For composite relations, the corresponding diagonal relation matrices can be non-commutative, reflecting a predominant scenario in real world applications; (2) Our model preserves the natural interaction between relational operations and entity embeddings; (3) The scaling operation provides the modeling power for the intrinsic semantic hierarchical structure of entities; (4) The enhanced expressiveness of DensE is achieved with high computational efficiency in terms of both parameter size and training time; and (5) Modeling entities in Euclidean space instead of quaternion space keeps the direct geometrical interpretations of relational patterns. Experimental results on multiple benchmark knowledge graphs show that DensE outperforms the current state-of-the-art models for missing link prediction, especially on composite relations.

Related papers

• Knowledge Graph Embeddings: A Comprehensive Survey on Capturing Relation Properties [5.651919225343915]
  Knowledge Graph Embedding (KGE) techniques play a pivotal role in transforming symbolic Knowledge Graphs into numerical representations. This paper addresses the complex mapping properties inherent in relations, such as one-to-one, one-to-many, many-to-one, and many-to-many mappings. We explore innovative ideas such as integrating multimodal information into KGE, enhancing relation pattern modeling with rules, and developing models to capture relation characteristics in dynamic KGE settings.
  arXiv  Detail & Related papers  (2024-10-16T08:54:52Z)
• Geometric Relational Embeddings [19.383110247906256]
  We propose relational embeddings, a paradigm of embeddings that respect the underlying symbolic structures. Results obtained from benchmark real-world datasets demonstrate the efficacy of geometric relational embeddings.
  arXiv  Detail & Related papers  (2024-09-18T22:02:24Z)
• Learning Complete Topology-Aware Correlations Between Relations for Inductive Link Prediction [121.65152276851619]
  We show that semantic correlations between relations are inherently edge-level and entity-independent. We propose a novel subgraph-based method, namely TACO, to model Topology-Aware COrrelations between relations. To further exploit the potential of RCN, we propose Complete Common Neighbor induced subgraph.
  arXiv  Detail & Related papers  (2023-09-20T08:11:58Z)
• A Comprehensive Study on Knowledge Graph Embedding over Relational Patterns Based on Rule Learning [49.09125100268454]
  Knowledge Graph Embedding (KGE) has proven to be an effective approach to solving the Knowledge Completion Graph (KGC) task. Relational patterns are an important factor in the performance of KGE models. We introduce a training-free method to enhance KGE models' performance over various relational patterns.
  arXiv  Detail & Related papers  (2023-08-15T17:30:57Z)
• 3D Rotation and Translation for Hyperbolic Knowledge Graph Embedding [6.453850986936394]
  This study introduces a novel model called 3H-TH (3D Rotation and Translation in Hyperbolic space) that captures relation patterns simultaneously. The experimental results demonstrate that the new model outperforms existing state-of-the-art models in terms of accuracy, hierarchy property, and other relation patterns in low-dimensional space.
  arXiv  Detail & Related papers  (2023-05-22T13:17:23Z)
• On Neural Architecture Inductive Biases for Relational Tasks [76.18938462270503]
  We introduce a simple architecture based on similarity-distribution scores which we name Compositional Network generalization (CoRelNet) We find that simple architectural choices can outperform existing models in out-of-distribution generalizations.
  arXiv  Detail & Related papers  (2022-06-09T16:24:01Z)
• STaR: Knowledge Graph Embedding by Scaling, Translation and Rotation [20.297699026433065]
  Bilinear method is mainstream in Knowledge Graph Embedding (KGE), aiming to learn low-dimensional representations for entities and relations. Previous works have mainly discovered 6 important patterns like non-commutativity. We propose a corresponding bilinear model Scaling Translation and Rotation (STaR) consisting of the above two parts.
  arXiv  Detail & Related papers  (2022-02-15T02:06:22Z)
• ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs [73.86041481470261]
  Cone Embeddings (ConE) is the first geometry-based query embedding model that can handle conjunction, disjunction, and negation. ConE significantly outperforms existing state-of-the-art methods on benchmark datasets.
  arXiv  Detail & Related papers  (2021-10-26T14:04:02Z)
• QuatDE: Dynamic Quaternion Embedding for Knowledge Graph Completion [4.837804582368272]
  We propose a novel model, QuatDE, with a dynamic mapping strategy to explicitly capture a variety of relational patterns. Experiment results show QuatDE achieves state-of-the-art performance on three well-established knowledge graph completion benchmarks.
  arXiv  Detail & Related papers  (2021-05-19T09:10:39Z)
• Unified Graph Structured Models for Video Understanding [93.72081456202672]
  We propose a message passing graph neural network that explicitly models relational-temporal relations. We show how our method is able to more effectively model relationships between relevant entities in the scene.
  arXiv  Detail & Related papers  (2021-03-29T14:37:35Z)
• RatE: Relation-Adaptive Translating Embedding for Knowledge Graph Completion [51.64061146389754]
  We propose a relation-adaptive translation function built upon a novel weighted product in complex space. We then present our Relation-adaptive translating Embedding (RatE) approach to score each graph triple.
  arXiv  Detail & Related papers  (2020-10-10T01:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.