The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey

• URL: http://arxiv.org/abs/2305.13854v1
• Date: Tue, 23 May 2023 09:22:33 GMT
• Title: The Evolution of Distributed Systems for Graph Neural Networks and their Origin in Graph Processing and Deep Learning: A Survey
• Authors: Jana Vatter, Ruben Mayer, Hans-Arno Jacobsen
• Abstract summary: Graph Neural Networks (GNNs) are an emerging research field. GNNs can be applied to various domains including recommendation systems, computer vision, natural language processing, biology and chemistry. We aim to fill this gap by summarizing and categorizing important methods and techniques for large-scale GNN solutions.
• Score: 17.746899445454048
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Neural Networks (GNNs) are an emerging research field. This specialized Deep Neural Network (DNN) architecture is capable of processing graph structured data and bridges the gap between graph processing and Deep Learning (DL). As graphs are everywhere, GNNs can be applied to various domains including recommendation systems, computer vision, natural language processing, biology and chemistry. With the rapid growing size of real world graphs, the need for efficient and scalable GNN training solutions has come. Consequently, many works proposing GNN systems have emerged throughout the past few years. However, there is an acute lack of overview, categorization and comparison of such systems. We aim to fill this gap by summarizing and categorizing important methods and techniques for large-scale GNN solutions. In addition, we establish connections between GNN systems, graph processing systems and DL systems.

Related papers

• Graphs Unveiled: Graph Neural Networks and Graph Generation [0.0]
  This paper provides a comprehensive overview of Graph Neural Networks (GNNs) We discuss the applications of graph neural networks across various domains. We present an advanced field in GNNs: graph generation.
  arXiv  Detail & Related papers  (2024-03-18T14:37:27Z)
• A Systematic Review of Deep Graph Neural Networks: Challenges, Classification, Architectures, Applications & Potential Utility in Bioinformatics [0.0]
  Graph neural networks (GNNs) employ message transmission between graph nodes to represent graph dependencies. GNNs have the potential to be an excellent tool for solving a wide range of biological challenges in bioinformatics research.
  arXiv  Detail & Related papers  (2023-11-03T10:25:47Z)
• Distributed Graph Neural Network Training: A Survey [51.77035975191926]
  Graph neural networks (GNNs) are a type of deep learning models that are trained on graphs and have been successfully applied in various domains. Despite the effectiveness of GNNs, it is still challenging for GNNs to efficiently scale to large graphs. As a remedy, distributed computing becomes a promising solution of training large-scale GNNs.
  arXiv  Detail & Related papers  (2022-11-01T01:57:00Z)
• Training Graph Neural Networks on Growing Stochastic Graphs [114.75710379125412]
  Graph Neural Networks (GNNs) rely on graph convolutions to exploit meaningful patterns in networked data. We propose to learn GNNs on very large graphs by leveraging the limit object of a sequence of growing graphs, the graphon.
  arXiv  Detail & Related papers  (2022-10-27T16:00:45Z)
• Automatic Relation-aware Graph Network Proliferation [182.30735195376792]
  We propose Automatic Relation-aware Graph Network Proliferation (ARGNP) for efficiently searching GNNs. These operations can extract hierarchical node/relational information and provide anisotropic guidance for message passing on a graph. Experiments on six datasets for four graph learning tasks demonstrate that GNNs produced by our method are superior to the current state-of-the-art hand-crafted and search-based GNNs.
  arXiv  Detail & Related papers  (2022-05-31T10:38:04Z)
• Wide and Deep Graph Neural Network with Distributed Online Learning [174.8221510182559]
  Graph neural networks (GNNs) are naturally distributed architectures for learning representations from network data. Online learning can be leveraged to retrain GNNs at testing time to overcome this issue. This paper develops the Wide and Deep GNN (WD-GNN), a novel architecture that can be updated with distributed online learning mechanisms.
  arXiv  Detail & Related papers  (2021-07-19T23:56:48Z)
• Increase and Conquer: Training Graph Neural Networks on Growing Graphs [116.03137405192356]
  We consider the problem of learning a graphon neural network (WNN) by training GNNs on graphs sampled Bernoulli from the graphon. Inspired by these results, we propose an algorithm to learn GNNs on large-scale graphs that, starting from a moderate number of nodes, successively increases the size of the graph during training.
  arXiv  Detail & Related papers  (2021-06-07T15:05:59Z)
• Analyzing the Performance of Graph Neural Networks with Pipe Parallelism [2.269587850533721]
  We focus on Graph Neural Networks (GNNs) that have found great success in tasks such as node or edge classification and link prediction. New approaches for processing larger networks are needed to advance graph techniques. We study how GNNs could be parallelized using existing tools and frameworks that are known to be successful in the deep learning community.
  arXiv  Detail & Related papers  (2020-12-20T04:20:38Z)
• AutoGraph: Automated Graph Neural Network [45.94642721490744]
  We propose a method to automate the deep Graph Neural Networks (GNNs) design. In our proposed method, we add a new type of skip connection to the GNNs search space to encourage feature reuse. We also allow our evolutionary algorithm to increase the layers of GNNs during the evolution to generate deeper networks.
  arXiv  Detail & Related papers  (2020-11-23T09:04:17Z)
• Computing Graph Neural Networks: A Survey from Algorithms to Accelerators [2.491032752533246]
  Graph Neural Networks (GNNs) have exploded onto the machine learning scene in recent years owing to their capability to model and learn from graph-structured data. This paper aims to make two main contributions: a review of the field of GNNs is presented from the perspective of computing. An in-depth analysis of current software and hardware acceleration schemes is provided.
  arXiv  Detail & Related papers  (2020-09-30T22:29:27Z)
• Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networks [183.97265247061847]
  We leverage graph signal processing to characterize the representation space of graph neural networks (GNNs) We discuss the role of graph convolutional filters in GNNs and show that any architecture built with such filters has the fundamental properties of permutation equivariance and stability to changes in the topology. We also study the use of GNNs in recommender systems and learning decentralized controllers for robot swarms.
  arXiv  Detail & Related papers  (2020-03-08T13:02:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.