Structure-reinforced Transformer for Dynamic Graph Representation Learning with Edge Temporal States
- URL: http://arxiv.org/abs/2304.10079v2
- Date: Wed, 3 Apr 2024 15:46:30 GMT
- Title: Structure-reinforced Transformer for Dynamic Graph Representation Learning with Edge Temporal States
- Authors: Shengxiang Hu, Guobing Zou, Song Yang, Shiyi Lin, Bofeng Zhang, Yixin Chen,
- Abstract summary: We introduce a novel dynamic graph representation learning framework namely Recurrent Structure-reinforced Graph Transformer (RSGT)
RSGT initially models the temporal status of edges explicitly by utilizing different edge types and weights based on the differences between any two consecutive snapshots.
A structure-reinforced graph transformer is proposed to capture temporal node representations that encoding both the graph topological structure and evolving dynamics.
- Score: 8.577434144370004
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The burgeoning field of dynamic graph representation learning, fuelled by the increasing demand for graph data analysis in real-world applications, poses both enticing opportunities and formidable challenges. Despite the promising results achieved by recent research leveraging recurrent neural networks (RNNs) and graph neural networks (GNNs), these approaches often fail to adequately consider the impact of the edge temporal states on the strength of inter-node relationships across different time slices, further overlooking the dynamic changes in node features induced by fluctuations in relationship strength. Furthermore, the extraction of global structural features is hindered by the inherent over-smoothing drawback of GNNs, which in turn limits their overall performance. In this paper, we introduce a novel dynamic graph representation learning framework namely Recurrent Structure-reinforced Graph Transformer (RSGT), which initially models the temporal status of edges explicitly by utilizing different edge types and weights based on the differences between any two consecutive snapshots. In this manner, the varying edge temporal states are mapped as a part of the topological structure of the graph. Subsequently, a structure-reinforced graph transformer is proposed to capture temporal node representations that encoding both the graph topological structure and evolving dynamics,through a recurrent learning paradigm. Our experimental evaluations, conducted on four real-world datasets, underscore the superior performance of the RSGT in the realm of discrete dynamic graph representation learning. The results reveal that RSGT consistently surpasses competing methods in dynamic link prediction tasks.
Related papers
- DURENDAL: Graph deep learning framework for temporal heterogeneous
networks [0.5156484100374057]
Temporal heterogeneous networks (THNs) are evolving networks that characterize many real-world applications.
We propose DURENDAL, a graph deep learning framework for THNs.
arXiv Detail & Related papers (2023-09-30T10:46:01Z) - Dynamic Causal Explanation Based Diffusion-Variational Graph Neural
Network for Spatio-temporal Forecasting [60.03169701753824]
We propose a novel Dynamic Diffusion-al Graph Neural Network (DVGNN) fortemporal forecasting.
The proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding Root Mean Squared Error result.
arXiv Detail & Related papers (2023-05-16T11:38:19Z) - Decoupled Graph Neural Networks for Large Dynamic Graphs [14.635923016087503]
We propose a decoupled graph neural network for large dynamic graphs.
We show that our algorithm achieves state-of-the-art performance in both kinds of dynamic graphs.
arXiv Detail & Related papers (2023-05-14T23:00:10Z) - Temporal Aggregation and Propagation Graph Neural Networks for Dynamic
Representation [67.26422477327179]
Temporal graphs exhibit dynamic interactions between nodes over continuous time.
We propose a novel method of temporal graph convolution with the whole neighborhood.
Our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency.
arXiv Detail & Related papers (2023-04-15T08:17:18Z) - EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph
Learning [114.72818205974285]
This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability.
EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
arXiv Detail & Related papers (2023-03-22T06:35:08Z) - Learning Dynamic Graph Embeddings with Neural Controlled Differential
Equations [21.936437653875245]
This paper focuses on representation learning for dynamic graphs with temporal interactions.
We propose a generic differential model for dynamic graphs that characterises the continuously dynamic evolution of node embedding trajectories.
Our framework exhibits several desirable characteristics, including the ability to express dynamics on evolving graphs without integration by segments.
arXiv Detail & Related papers (2023-02-22T12:59:38Z) - Time-aware Dynamic Graph Embedding for Asynchronous Structural Evolution [60.695162101159134]
Existing works merely view a dynamic graph as a sequence of changes.
We formulate dynamic graphs as temporal edge sequences associated with joining time of.
vertex and timespan of edges.
A time-aware Transformer is proposed to embed.
vertex' dynamic connections and ToEs into the learned.
vertex representations.
arXiv Detail & Related papers (2022-07-01T15:32:56Z) - Efficient Dynamic Graph Representation Learning at Scale [66.62859857734104]
We propose Efficient Dynamic Graph lEarning (EDGE), which selectively expresses certain temporal dependency via training loss to improve the parallelism in computations.
We show that EDGE can scale to dynamic graphs with millions of nodes and hundreds of millions of temporal events and achieve new state-of-the-art (SOTA) performance.
arXiv Detail & Related papers (2021-12-14T22:24:53Z) - ConvDySAT: Deep Neural Representation Learning on Dynamic Graphs via
Self-Attention and Convolutional Neural Networks [0.0]
We present ConvDySAT as an enhancement of DySAT, one of the state-of-the-art dynamic methods, by augmenting convolution neural networks with the self-attention mechanism.
We conducted single-step link prediction on a communication network and rating network, Experimental results show significant performance gains for ConvDySAT over various state-of-the-art methods.
arXiv Detail & Related papers (2021-06-21T22:13:20Z) - Anomaly Detection in Dynamic Graphs via Transformer [30.926884264054042]
We present a novel Transformer-based Anomaly Detection framework for DYnamic graph (TADDY)
Our framework constructs a comprehensive node encoding strategy to better represent each node's structural and temporal roles in an evolving graphs stream.
Our proposed TADDY framework outperforms the state-of-the-art methods by a large margin on four real-world datasets.
arXiv Detail & Related papers (2021-06-18T02:27:19Z) - TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning [87.38675639186405]
We propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion.
To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs.
arXiv Detail & Related papers (2021-05-17T15:33:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.