PIE: a Parameter and Inference Efficient Solution for Large Scale Knowledge Graph Embedding Reasoning

• URL: http://arxiv.org/abs/2204.13957v1
• Date: Fri, 29 Apr 2022 09:06:56 GMT
• Title: PIE: a Parameter and Inference Efficient Solution for Large Scale Knowledge Graph Embedding Reasoning
• Authors: Linlin Chao, Taifeng Wang, Wei Chu
• Abstract summary: We propose PIE, a textbfparameter and textbfinference textbfefficient solution. Inspired from tensor decomposition methods, we find that decompose entity embedding matrix into low rank matrices can reduce more than half of the parameters. To accelerate model inference, we propose a self-supervised auxiliary task, which can be seen as fine-grained entity typing.
• Score: 24.29409958504209
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge graph (KG) embedding methods which map entities and relations to unique embeddings in the KG have shown promising results on many reasoning tasks. However, the same embedding dimension for both dense entities and sparse entities will cause either over parameterization (sparse entities) or under fitting (dense entities). Normally, a large dimension is set to get better performance. Meanwhile, the inference time grows log-linearly with the number of entities for all entities are traversed and compared. Both the parameter and inference become challenges when working with huge amounts of entities. Thus, we propose PIE, a \textbf{p}arameter and \textbf{i}nference \textbf{e}fficient solution. Inspired from tensor decomposition methods, we find that decompose entity embedding matrix into low rank matrices can reduce more than half of the parameters while maintaining comparable performance. To accelerate model inference, we propose a self-supervised auxiliary task, which can be seen as fine-grained entity typing. By randomly masking and recovering entities' connected relations, the task learns the co-occurrence of entity and relations. Utilizing the fine grained typing, we can filter unrelated entities during inference and get targets with possibly sub-linear time requirement. Experiments on link prediction benchmarks demonstrate the proposed key capabilities. Moreover, we prove effectiveness of the proposed solution on the Open Graph Benchmark large scale challenge dataset WikiKG90Mv2 and achieve the state of the art performance.

Related papers

• Entity Disambiguation via Fusion Entity Decoding [68.77265315142296]
  We propose an encoder-decoder model to disambiguate entities with more detailed entity descriptions. We observe +1.5% improvements in end-to-end entity linking in the GERBIL benchmark compared with EntQA.
  arXiv  Detail & Related papers  (2024-04-02T04:27:54Z)
• EventEA: Benchmarking Entity Alignment for Event-centric Knowledge Graphs [17.27027602556303]
  We show that the progress made in the past was due to biased and unchallenging evaluation. We construct a new dataset with heterogeneous relations and attributes based on event-centric KGs. As a new approach to this difficult problem, we propose a time-aware literal encoder for entity alignment.
  arXiv  Detail & Related papers  (2022-11-05T05:34:21Z)
• Explainable Sparse Knowledge Graph Completion via High-order Graph Reasoning Network [111.67744771462873]
  This paper proposes a novel explainable model for sparse Knowledge Graphs (KGs) It combines high-order reasoning into a graph convolutional network, namely HoGRN. It can not only improve the generalization ability to mitigate the information insufficiency issue but also provide interpretability.
  arXiv  Detail & Related papers  (2022-07-14T10:16:56Z)
• Probabilistic Entity Representation Model for Chain Reasoning over Knowledge Graphs [18.92547855877845]
  We propose a Probabilistic Entity Representation Model (PERM) for logical reasoning over Knowledge Graphs. PERM encodes entities as a Multivariate Gaussian density with mean and covariance parameters to capture semantic position and smooth decision boundary. We demonstrate PERM's competence on a COVID-19 drug-repurposing case study and show that our proposed work is able to recommend drugs with substantially better F1 than current methods.
  arXiv  Detail & Related papers  (2021-10-26T09:26:10Z)
• Entity Linking and Discovery via Arborescence-based Supervised Clustering [35.93568319872986]
  We present novel training and inference procedures that fully utilize mention-to-mention affinities. We show that this method gracefully extends to entity discovery. We evaluate our approach on the Zero-Shot Entity Linking dataset and MedMentions, the largest publicly available biomedical dataset.
  arXiv  Detail & Related papers  (2021-09-02T23:05:58Z)
• EchoEA: Echo Information between Entities and Relations for Entity Alignment [1.1470070927586016]
  We propose a novel framework, Echo Entity Alignment (EchoEA), which leverages self-attention mechanism to spread entity information to relations and echo back to entities. The experimental results on three real-world cross-lingual datasets are stable at around 96% at hits@1 on average.
  arXiv  Detail & Related papers  (2021-07-07T07:34:21Z)
• Unsupervised Knowledge Graph Alignment by Probabilistic Reasoning and Semantic Embedding [22.123001954919893]
  We propose an iterative framework named PRASE which is based on probabilistic reasoning and semantic embedding. The PRASE framework is compatible with different embedding-based models, and our experiments on multiple datasets have demonstrated its state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-12T11:27:46Z)
• Cross-Supervised Joint-Event-Extraction with Heterogeneous Information Networks [61.950353376870154]
  Joint-event-extraction is a sequence-to-sequence labeling task with a tag set composed of tags of triggers and entities. We propose a Cross-Supervised Mechanism (CSM) to alternately supervise the extraction of triggers or entities. Our approach outperforms the state-of-the-art methods in both entity and trigger extraction.
  arXiv  Detail & Related papers  (2020-10-13T11:51:17Z)
• Joint Semantics and Data-Driven Path Representation for Knowledge Graph Inference [60.048447849653876]
  We propose a novel joint semantics and data-driven path representation that balances explainability and generalization in the framework of KG embedding. Our proposed model is evaluated on two classes of tasks: link prediction and path query answering task.
  arXiv  Detail & Related papers  (2020-10-06T10:24:45Z)
• Autoregressive Entity Retrieval [55.38027440347138]
  Entities are at the center of how we represent and aggregate knowledge. The ability to retrieve such entities given a query is fundamental for knowledge-intensive tasks such as entity linking and open-domain question answering. We propose GENRE, the first system that retrieves entities by generating their unique names, left to right, token-by-token in an autoregressive fashion.
  arXiv  Detail & Related papers  (2020-10-02T10:13:31Z)
• Polynomial-Time Exact MAP Inference on Discrete Models with Global Dependencies [83.05591911173332]
  junction tree algorithm is the most general solution for exact MAP inference with run-time guarantees. We propose a new graph transformation technique via node cloning which ensures a run-time for solving our target problem independently of the form of a corresponding clique tree.
  arXiv  Detail & Related papers  (2019-12-27T13:30:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.