Transformer-based Reasoning for Learning Evolutionary Chain of Events on Temporal Knowledge Graph

• URL: http://arxiv.org/abs/2405.00352v1
• Date: Wed, 1 May 2024 07:12:16 GMT
• Title: Transformer-based Reasoning for Learning Evolutionary Chain of Events on Temporal Knowledge Graph
• Authors: Zhiyu Fang, Shuai-Long Lei, Xiaobin Zhu, Chun Yang, Shi-Xue Zhang, Xu-Cheng Yin, Jingyan Qin,
• Abstract summary: Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. We propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE) We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE.
• Score: 22.427652636877774
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal Knowledge Graph (TKG) reasoning often involves completing missing factual elements along the timeline. Although existing methods can learn good embeddings for each factual element in quadruples by integrating temporal information, they often fail to infer the evolution of temporal facts. This is mainly because of (1) insufficiently exploring the internal structure and semantic relationships within individual quadruples and (2) inadequately learning a unified representation of the contextual and temporal correlations among different quadruples. To overcome these limitations, we propose a novel Transformer-based reasoning model (dubbed ECEformer) for TKG to learn the Evolutionary Chain of Events (ECE). Specifically, we unfold the neighborhood subgraph of an entity node in chronological order, forming an evolutionary chain of events as the input for our model. Subsequently, we utilize a Transformer encoder to learn the embeddings of intra-quadruples for ECE. We then craft a mixed-context reasoning module based on the multi-layer perceptron (MLP) to learn the unified representations of inter-quadruples for ECE while accomplishing temporal knowledge reasoning. In addition, to enhance the timeliness of the events, we devise an additional time prediction task to complete effective temporal information within the learned unified representation. Extensive experiments on six benchmark datasets verify the state-of-the-art performance and the effectiveness of our method.

Related papers

• CTRL: Continuous-Time Representation Learning on Temporal Heterogeneous Information Network [32.42051167404171]
  We propose a Continuous-Time Representation Learning model on temporal HINs. We train the model with a future event (a subgraph) prediction task to capture the evolution of the high-order network structure. The results demonstrate that our model significantly boosts performance and outperforms various state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-05-11T03:39:22Z)
• Arbitrary Time Information Modeling via Polynomial Approximation for Temporal Knowledge Graph Embedding [23.39851202825318]
  temporal knowledge graphs (TKGs) must explore and reason over temporally evolving facts adequately. Existing TKG approaches face two main challenges, i.e., the limited capability to model arbitrary timestamps continuously and the lack of rich inference patterns under temporal constraints. We propose an innovative TKGE method (PTBox) via decomposition-based temporal representation and embedding-based entity representation.
  arXiv  Detail & Related papers  (2024-05-01T07:27:04Z)
• Time-Aware Knowledge Representations of Dynamic Objects with Multidimensional Persistence [41.32931849366751]
  We propose a new approach to a time-aware knowledge representation mechanism that focuses on implicit time-dependent topological information. In particular, we propose a new approach, named textitTemporal MultiPersistence (TMP), which produces multidimensional topological fingerprints of the data. TMP method improves the computational efficiency of the state-of-the-art multipersistence summaries up to 59.5 times.
  arXiv  Detail & Related papers  (2024-01-24T00:33:53Z)
• Spatial-Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation [64.85974098314344]
  Video scene graph generation (VidSGG) aims to identify objects in visual scenes and infer their relationships for a given video. Inherently, object pairs and their relationships enjoy spatial co-occurrence correlations within each image and temporal consistency/transition correlations across different images. We propose a spatial-temporal knowledge-embedded transformer (STKET) that incorporates the prior spatial-temporal knowledge into the multi-head cross-attention mechanism.
  arXiv  Detail & Related papers  (2023-09-23T02:40:28Z)
• Deeply-Coupled Convolution-Transformer with Spatial-temporal Complementary Learning for Video-based Person Re-identification [91.56939957189505]
  We propose a novel spatial-temporal complementary learning framework named Deeply-Coupled Convolution-Transformer (DCCT) for high-performance video-based person Re-ID. Our framework could attain better performances than most state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-27T12:16:44Z)
• Generic Temporal Reasoning with Differential Analysis and Explanation [61.96034987217583]
  We introduce a novel task named TODAY that bridges the gap with temporal differential analysis. TODAY evaluates whether systems can correctly understand the effect of incremental changes. We show that TODAY's supervision style and explanation annotations can be used in joint learning.
  arXiv  Detail & Related papers  (2022-12-20T17:40:03Z)
• DyTed: Disentangled Representation Learning for Discrete-time Dynamic Graph [59.583555454424]
  We propose a novel disenTangled representation learning framework for discrete-time Dynamic graphs, namely DyTed. We specially design a temporal-clips contrastive learning task together with a structure contrastive learning to effectively identify the time-invariant and time-varying representations respectively.
  arXiv  Detail & Related papers  (2022-10-19T14:34:12Z)
• ECOLA: Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations [35.51427298619691]
  We study enhancing temporal knowledge embedding with textual data. We propose Enhanced Temporal Knowledge Embeddings with Contextualized Language Representations (ECOLA) Experiments show that ECOLA significantly enhances temporal embedding models with up to 287% relative improvements regarding Hits@1 on the link prediction task.
  arXiv  Detail & Related papers  (2022-03-17T20:08:25Z)
• Complex Evolutional Pattern Learning for Temporal Knowledge Graph Reasoning [60.94357727688448]
  TKG reasoning aims to predict potential facts in the future given the historical KG sequences. The evolutional patterns are complex in two aspects, length-diversity and time-variability. We propose a new model, called Complex Evolutional Network (CEN), which uses a length-aware Convolutional Neural Network (CNN) to handle evolutional patterns of different lengths.
  arXiv  Detail & Related papers  (2022-03-15T11:02:55Z)
• Interpretable Time-series Representation Learning With Multi-Level Disentanglement [56.38489708031278]
  Disentangle Time Series (DTS) is a novel disentanglement enhancement framework for sequential data. DTS generates hierarchical semantic concepts as the interpretable and disentangled representation of time-series. DTS achieves superior performance in downstream applications, with high interpretability of semantic concepts.
  arXiv  Detail & Related papers  (2021-05-17T22:02:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.