Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective

• URL: http://arxiv.org/abs/2311.13279v2
• Date: Wed, 20 Mar 2024 02:25:36 GMT
• Title: Comprehensive Evaluation of GNN Training Systems: A Data Management Perspective
• Authors: Hao Yuan, Yajiong Liu, Yanfeng Zhang, Xin Ai, Qiange Wang, Chaoyi Chen, Yu Gu, Ge Yu,
• Abstract summary: Many Graph Neural Network (GNN) training systems have emerged recently to support efficient GNN training. This paper reviews GNN training from a data management perspective and provides a comprehensive analysis and evaluation of the representative approaches.
• Score: 18.83907327497481
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many Graph Neural Network (GNN) training systems have emerged recently to support efficient GNN training. Since GNNs embody complex data dependencies between training samples, the training of GNNs should address distinct challenges different from DNN training in data management, such as data partitioning, batch preparation for mini-batch training, and data transferring between CPUs and GPUs. These factors, which take up a large proportion of training time, make data management in GNN training more significant. This paper reviews GNN training from a data management perspective and provides a comprehensive analysis and evaluation of the representative approaches. We conduct extensive experiments on various benchmark datasets and show many interesting and valuable results. We also provide some practical tips learned from these experiments, which are helpful for designing GNN training systems in the future.

Related papers

• Stealing Training Graphs from Graph Neural Networks [54.52392250297907]
  Graph Neural Networks (GNNs) have shown promising results in modeling graphs in various tasks. As neural networks can memorize the training samples, the model parameters of GNNs have a high risk of leaking private training data. We investigate a novel problem of stealing graphs from trained GNNs.
  arXiv  Detail & Related papers  (2024-11-17T23:15:36Z)
• Label Deconvolution for Node Representation Learning on Large-scale Attributed Graphs against Learning Bias [75.44877675117749]
  We propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph datasets Benchmark.
  arXiv  Detail & Related papers  (2023-09-26T13:09:43Z)
• A Comprehensive Survey on Distributed Training of Graph Neural Networks [59.785830738482474]
  Graph neural networks (GNNs) have been demonstrated to be a powerful algorithmic model in broad application fields. To scale GNN training up for large-scale and ever-growing graphs, the most promising solution is distributed training. The volume of related research on distributed GNN training is exceptionally vast, accompanied by an extraordinarily rapid pace of publication.
  arXiv  Detail & Related papers  (2022-11-10T06:22:12Z)
• Distributed Graph Neural Network Training: A Survey [51.77035975191926]
  Graph neural networks (GNNs) are a type of deep learning models that are trained on graphs and have been successfully applied in various domains. Despite the effectiveness of GNNs, it is still challenging for GNNs to efficiently scale to large graphs. As a remedy, distributed computing becomes a promising solution of training large-scale GNNs.
  arXiv  Detail & Related papers  (2022-11-01T01:57:00Z)
• Characterizing and Understanding Distributed GNN Training on GPUs [2.306379679349986]
  Graph neural network (GNN) has been demonstrated to be a powerful model in many domains for its effectiveness in learning over graphs. To scale GNN training for large graphs, a widely adopted approach is distributed training which accelerates training using multiple computing nodes.
  arXiv  Detail & Related papers  (2022-04-18T03:47:28Z)
• Shift-Robust GNNs: Overcoming the Limitations of Localized Graph Training data [52.771780951404565]
  Shift-Robust GNN (SR-GNN) is designed to account for distributional differences between biased training data and the graph's true inference distribution. We show that SR-GNN outperforms other GNN baselines by accuracy, eliminating at least (40%) of the negative effects introduced by biased training data.
  arXiv  Detail & Related papers  (2021-08-02T18:00:38Z)
• FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks [68.64678614325193]
  Graph Neural Network (GNN) research is rapidly growing thanks to the capacity of GNNs to learn representations from graph-structured data. Centralizing a massive amount of real-world graph data for GNN training is prohibitive due to user-side privacy concerns. We introduce FedGraphNN, an open research federated learning system and a benchmark to facilitate GNN-based FL research.
  arXiv  Detail & Related papers  (2021-04-14T22:11:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.