CPDG: A Contrastive Pre-Training Method for Dynamic Graph Neural Networks

• URL: http://arxiv.org/abs/2307.02813v3
• Date: Sun, 24 Dec 2023 05:56:49 GMT
• Title: CPDG: A Contrastive Pre-Training Method for Dynamic Graph Neural Networks
• Authors: Yuanchen Bei, Hao Xu, Sheng Zhou, Huixuan Chi, Haishuai Wang, Mengdi Zhang, Zhao Li, Jiajun Bu
• Abstract summary: We propose Contrastive Pre-Training Method for Dynamic Graph Neural Networks (CPDG) CPDG tackles the challenges of pre-training for DGNNs, including generalization capability and long-short term modeling capability. Extensive experiments conducted on both large-scale research and industrial dynamic graph datasets.
• Score: 21.79251709065902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dynamic graph data mining has gained popularity in recent years due to the rich information contained in dynamic graphs and their widespread use in the real world. Despite the advances in dynamic graph neural networks (DGNNs), the rich information and diverse downstream tasks have posed significant difficulties for the practical application of DGNNs in industrial scenarios. To this end, in this paper, we propose to address them by pre-training and present the Contrastive Pre-Training Method for Dynamic Graph Neural Networks (CPDG). CPDG tackles the challenges of pre-training for DGNNs, including generalization capability and long-short term modeling capability, through a flexible structural-temporal subgraph sampler along with structural-temporal contrastive pre-training schemes. Extensive experiments conducted on both large-scale research and industrial dynamic graph datasets show that CPDG outperforms existing methods in dynamic graph pre-training for various downstream tasks under three transfer settings.

Related papers

• Gradient Transformation: Towards Efficient and Model-Agnostic Unlearning for Dynamic Graph Neural Networks [66.70786325911124]
  Graph unlearning has emerged as an essential tool for safeguarding user privacy and mitigating the negative impacts of undesirable data. With the increasing prevalence of DGNNs, it becomes imperative to investigate the implementation of dynamic graph unlearning. We propose an effective, efficient, model-agnostic, and post-processing method to implement DGNN unlearning.
  arXiv  Detail & Related papers  (2024-05-23T10:26:18Z)
• Node-Time Conditional Prompt Learning In Dynamic Graphs [14.62182210205324]
  We propose DYGPROMPT, a novel pre-training and prompt learning framework for dynamic graph modeling. We recognize that node and time features mutually characterize each other, and propose dual condition-nets to model the evolving node-time patterns in downstream tasks.
  arXiv  Detail & Related papers  (2024-05-22T19:10:24Z)
• A survey of dynamic graph neural networks [26.162035361191805]
  Graph neural networks (GNNs) have emerged as a powerful tool for effectively mining and learning from graph-structured data. This paper provides a comprehensive review of the fundamental concepts, key techniques, and state-of-the-art dynamic GNN models.
  arXiv  Detail & Related papers  (2024-04-28T15:07:48Z)
• Dynamic Graph Representation Learning with Neural Networks: A Survey [0.0]
  Dynamic graph representations have emerged as a new machine learning problem. This paper aims at providing a review of problems and models related to dynamic graph learning.
  arXiv  Detail & Related papers  (2023-04-12T09:39:17Z)
• EasyDGL: Encode, Train and Interpret for Continuous-time Dynamic Graph Learning [92.71579608528907]
  This paper aims to design an easy-to-use pipeline (termed as EasyDGL) composed of three key modules with both strong ability fitting and interpretability. EasyDGL can effectively quantify the predictive power of frequency content that a model learn from the evolving graph data.
  arXiv  Detail & Related papers  (2023-03-22T06:35:08Z)
• MentorGNN: Deriving Curriculum for Pre-Training GNNs [61.97574489259085]
  We propose an end-to-end model named MentorGNN that aims to supervise the pre-training process of GNNs across graphs. We shed new light on the problem of domain adaption on relational data (i.e., graphs) by deriving a natural and interpretable upper bound on the generalization error of the pre-trained GNNs.
  arXiv  Detail & Related papers  (2022-08-21T15:12:08Z)
• Data-Free Adversarial Knowledge Distillation for Graph Neural Networks [62.71646916191515]
  We propose the first end-to-end framework for data-free adversarial knowledge distillation on graph structured data (DFAD-GNN) To be specific, our DFAD-GNN employs a generative adversarial network, which mainly consists of three components: a pre-trained teacher model and a student model are regarded as two discriminators, and a generator is utilized for deriving training graphs to distill knowledge from the teacher model into the student model. Our DFAD-GNN significantly surpasses state-of-the-art data-free baselines in the graph classification task.
  arXiv  Detail & Related papers  (2022-05-08T08:19:40Z)
• Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs [77.33781731432163]
  We learn dynamic graph representation in hyperbolic space, for the first time, which aims to infer node representations. We present a novel Hyperbolic Variational Graph Network, referred to as HVGNN. In particular, to model the dynamics, we introduce a Temporal GNN (TGNN) based on a theoretically grounded time encoding approach.
  arXiv  Detail & Related papers  (2021-04-06T01:44:15Z)
• Pre-Training on Dynamic Graph Neural Networks [26.139844652756334]
  This paper proposes a pre-training method on dynamic graph neural networks (PT-DGNN) It uses dynamic attributed graph generation tasks to simultaneously learn the structure, semantics, and evolution features of the graph.
  arXiv  Detail & Related papers  (2021-02-24T16:06:32Z)
• GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training [62.73470368851127]
  Graph representation learning has emerged as a powerful technique for addressing real-world problems. We design Graph Contrastive Coding -- a self-supervised graph neural network pre-training framework. We conduct experiments on three graph learning tasks and ten graph datasets.
  arXiv  Detail & Related papers  (2020-06-17T16:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.