zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models

• URL: http://arxiv.org/abs/2311.10112v2
• Date: Fri, 15 Mar 2024 15:38:07 GMT
• Title: zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
• Authors: Zifeng Ding, Heling Cai, Jingpei Wu, Yunpu Ma, Ruotong Liao, Bo Xiong, Volker Tresp,
• Abstract summary: We use large language models to generate relation representations for embedding-based TKGF methods. We show that our approach helps TKGF models to achieve much better performance in forecasting the facts with previously unseen relations.
• Score: 33.10218179341504
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modeling evolving knowledge over temporal knowledge graphs (TKGs) has become a heated topic. Various methods have been proposed to forecast links on TKGs. Most of them are embedding-based, where hidden representations are learned to represent knowledge graph (KG) entities and relations based on the observed graph contexts. Although these methods show strong performance on traditional TKG forecasting (TKGF) benchmarks, they face a strong challenge in modeling the unseen zero-shot relations that have no prior graph context. In this paper, we try to mitigate this problem as follows. We first input the text descriptions of KG relations into large language models (LLMs) for generating relation representations, and then introduce them into embedding-based TKGF methods. LLM-empowered representations can capture the semantic information in the relation descriptions. This makes the relations, whether seen or unseen, with similar semantic meanings stay close in the embedding space, enabling TKGF models to recognize zero-shot relations even without any observed graph context. Experimental results show that our approach helps TKGF models to achieve much better performance in forecasting the facts with previously unseen relations, while still maintaining their ability in link forecasting regarding seen relations.

Related papers

• Learning Complete Topology-Aware Correlations Between Relations for Inductive Link Prediction [121.65152276851619]
  We show that semantic correlations between relations are inherently edge-level and entity-independent. We propose a novel subgraph-based method, namely TACO, to model Topology-Aware COrrelations between relations. To further exploit the potential of RCN, we propose Complete Common Neighbor induced subgraph.
  arXiv  Detail & Related papers  (2023-09-20T08:11:58Z)
• Knowledge Graph Completion with Counterfactual Augmentation [23.20561746976504]
  We introduce a counterfactual question: "would the relation still exist if the neighborhood of entities became different from observation?" With a carefully designed instantiation of a causal model on the knowledge graph, we generate the counterfactual relations to answer the question. We incorporate the created counterfactual relations with the GNN-based framework on KGs to augment their learning of entity pair representations.
  arXiv  Detail & Related papers  (2023-02-25T14:08:15Z)
• Meta-Learning Based Knowledge Extrapolation for Temporal Knowledge Graph [4.103806361930888]
  Temporal KGs (TKGs) extend traditional Knowledge Graphs by associating static triples with timestamps forming quadruples. We propose a Meta-Learning based Temporal Knowledge Graph Extrapolation (MTKGE) model, which is trained on link prediction tasks sampled from the existing TKGs. We show that MTKGE consistently outperforms both the existing state-of-the-art models for knowledge graph extrapolation.
  arXiv  Detail & Related papers  (2023-02-11T09:52:26Z)
• Learning Meta Representations of One-shot Relations for Temporal Knowledge Graph Link Prediction [33.36701435886095]
  Few-shot relational learning for static knowledge graphs (KGs) has drawn greater interest in recent years. TKGs contain rich temporal information, thus requiring temporal reasoning techniques for modeling. This poses a greater challenge in learning few-shot relations in the temporal context.
  arXiv  Detail & Related papers  (2022-05-21T15:17:52Z)
• TranS: Transition-based Knowledge Graph Embedding with Synthetic Relation Representation [14.759663752868487]
  We propose a novel transition-based method, TranS, for knowledge graph embedding. The single relation vector in traditional scoring patterns is replaced with synthetic relation representation, which can solve these issues effectively and efficiently. Experiments on a large knowledge graph dataset, ogbl-wikikg2, show that our model achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2022-04-18T16:55:25Z)
• Learning Representations of Entities and Relations [0.0]
  This thesis focuses on improving knowledge graph representation with the aim of tackling the link prediction task. The first contribution is HypER, a convolutional model which simplifies and improves upon the link prediction performance. The second contribution is TuckER, a relatively straightforward linear model, which, at the time of its introduction, obtained state-of-the-art link prediction performance. The third contribution is MuRP, first multi-relational graph representation model embedded in hyperbolic space.
  arXiv  Detail & Related papers  (2022-01-31T09:24:43Z)
• RelWalk A Latent Variable Model Approach to Knowledge Graph Embedding [50.010601631982425]
  This paper extends the random walk model (Arora et al., 2016a) of word embeddings to Knowledge Graph Embeddings (KGEs) We derive a scoring function that evaluates the strength of a relation R between two entities h (head) and t (tail) We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph.
  arXiv  Detail & Related papers  (2021-01-25T13:31:29Z)
• Tensor Composition Net for Visual Relationship Prediction [115.14829858763399]
  We present a novel Composition Network (TCN) to predict visual relationships in images. The key idea of our TCN is to exploit the low rank property of the visual relationship tensor. We show our TCN's image-level visual relationship prediction provides a simple and efficient mechanism for relation-based image retrieval.
  arXiv  Detail & Related papers  (2020-12-10T06:27:20Z)
• Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction [84.64435075778988]
  We propose a general approach to learn relation prototypes from unlabeled texts. We learn relation prototypes as an implicit factor between entities. We conduct experiments on two publicly available datasets: New York Times and Google Distant Supervision.
  arXiv  Detail & Related papers  (2020-11-27T06:21:12Z)
• One-shot Learning for Temporal Knowledge Graphs [49.41854171118697]
  We propose a one-shot learning framework for link prediction in temporal knowledge graphs. Our proposed method employs a self-attention mechanism to effectively encode temporal interactions between entities. Our experiments show that the proposed algorithm outperforms the state of the art baselines for two well-studied benchmarks.
  arXiv  Detail & Related papers  (2020-10-23T03:24:44Z)
• Generative Adversarial Zero-Shot Relational Learning for Knowledge Graphs [96.73259297063619]
  We consider a novel formulation, zero-shot learning, to free this cumbersome curation. For newly-added relations, we attempt to learn their semantic features from their text descriptions. We leverage Generative Adrial Networks (GANs) to establish the connection between text and knowledge graph domain.
  arXiv  Detail & Related papers  (2020-01-08T01:19:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.