D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks

• URL: http://arxiv.org/abs/2409.09079v1
• Date: Tue, 10 Sep 2024 11:00:43 GMT
• Title: D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks
• Authors: Rustam Guliyev, Aparajita Haldar, Hakan Ferhatosmanoglu,
• Abstract summary: Graph Neural Network (GNN) models on streaming graphs entail algorithmic challenges to continuously capture its dynamic state. We present D3-GNN, the first distributed, hybrid-parallel, streaming GNN system designed to handle real-time graph updates under online query setting.
• Score: 2.3283463706065763
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Neural Network (GNN) models on streaming graphs entail algorithmic challenges to continuously capture its dynamic state, as well as systems challenges to optimize latency, memory, and throughput during both inference and training. We present D3-GNN, the first distributed, hybrid-parallel, streaming GNN system designed to handle real-time graph updates under online query setting. Our system addresses data management, algorithmic, and systems challenges, enabling continuous capturing of the dynamic state of the graph and updating node representations with fault-tolerance and optimal latency, load-balance, and throughput. D3-GNN utilizes streaming GNN aggregators and an unrolled, distributed computation graph architecture to handle cascading graph updates. To counteract data skew and neighborhood explosion issues, we introduce inter-layer and intra-layer windowed forward pass solutions. Experiments on large-scale graph streams demonstrate that D3-GNN achieves high efficiency and scalability. Compared to DGL, D3-GNN achieves a significant throughput improvement of about 76x for streaming workloads. The windowed enhancement further reduces running times by around 10x and message volumes by up to 15x at higher parallelism.

Related papers

• NeutronStream: A Dynamic GNN Training Framework with Sliding Window for Graph Streams [12.365456024506901]
  NeutronStream is a framework for training dynamic Graph Neural Network (GNN) models. It captures both the spatial and temporal dependencies of graph updates. NeutronStream achieves speedups ranging from 1.48X to 5.87X and an average accuracy improvement of 3.97%.
  arXiv  Detail & Related papers  (2023-12-05T03:58:05Z)
• GNNFlow: A Distributed Framework for Continuous Temporal GNN Learning on Dynamic Graphs [11.302970701867844]
  We introduce GNNFlow, a distributed framework for efficient continuous temporal graph representation learning. GNNFlow supports distributed training across multiple machines with static scheduling to ensure load balance. Our experimental results show that GNNFlow provides up to 21.1x faster continuous learning than existing systems.
  arXiv  Detail & Related papers  (2023-11-29T07:30:32Z)
• InkStream: Real-time GNN Inference on Streaming Graphs via Incremental Update [6.4677254491956955]
  Classic Graph Neural Network (GNN) inference approaches, designed for static graphs, are ill-suited for streaming graphs that evolve with time. We propose a novel method, InkStream, designed for real-time inference with minimal memory access and computation. Our experiments with three GNN models on four large graphs demonstrate that InkStream accelerates by 2.5-427$times$ on a CPU cluster and 2.4-343$times$ on two different GPU clusters.
  arXiv  Detail & Related papers  (2023-09-20T05:34:52Z)
• Dynamic Causal Explanation Based Diffusion-Variational Graph Neural Network for Spatio-temporal Forecasting [60.03169701753824]
  We propose a novel Dynamic Diffusion-al Graph Neural Network (DVGNN) fortemporal forecasting. The proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding Root Mean Squared Error result.
  arXiv  Detail & Related papers  (2023-05-16T11:38:19Z)
• Temporal Aggregation and Propagation Graph Neural Networks for Dynamic Representation [67.26422477327179]
  Temporal graphs exhibit dynamic interactions between nodes over continuous time. We propose a novel method of temporal graph convolution with the whole neighborhood. Our proposed TAP-GNN outperforms existing temporal graph methods by a large margin in terms of both predictive performance and online inference latency.
  arXiv  Detail & Related papers  (2023-04-15T08:17:18Z)
• Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
  Modelling long-range dependencies is critical for scene understanding tasks in computer vision. A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
  arXiv  Detail & Related papers  (2022-09-20T14:41:37Z)
• Instant Graph Neural Networks for Dynamic Graphs [18.916632816065935]
  We propose Instant Graph Neural Network (InstantGNN), an incremental approach for the graph representation matrix of dynamic graphs. Our method avoids time-consuming, repetitive computations and allows instant updates on the representation and instant predictions. Our model achieves state-of-the-art accuracy while having orders-of-magnitude higher efficiency than existing methods.
  arXiv  Detail & Related papers  (2022-06-03T03:27:42Z)
• Increase and Conquer: Training Graph Neural Networks on Growing Graphs [116.03137405192356]
  We consider the problem of learning a graphon neural network (WNN) by training GNNs on graphs sampled Bernoulli from the graphon. Inspired by these results, we propose an algorithm to learn GNNs on large-scale graphs that, starting from a moderate number of nodes, successively increases the size of the graph during training.
  arXiv  Detail & Related papers  (2021-06-07T15:05:59Z)
• Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solution [18.309299822858243]
  We propose a novel traffic prediction framework, named Dynamic Graph Contemporalal Recurrent Network (DGCRN) In DGCRN, hyper-networks are designed to leverage and extract dynamic characteristics from node attributes. We are the first to employ a generation method to model fine iteration of dynamic graph at each time step.
  arXiv  Detail & Related papers  (2021-04-30T11:25:43Z)
• Hyperbolic Variational Graph Neural Network for Modeling Dynamic Graphs [77.33781731432163]
  We learn dynamic graph representation in hyperbolic space, for the first time, which aims to infer node representations. We present a novel Hyperbolic Variational Graph Network, referred to as HVGNN. In particular, to model the dynamics, we introduce a Temporal GNN (TGNN) based on a theoretically grounded time encoding approach.
  arXiv  Detail & Related papers  (2021-04-06T01:44:15Z)
• Graph Highway Networks [77.38665506495553]
  Graph Convolution Networks (GCN) are widely used in learning graph representations due to their effectiveness and efficiency. They suffer from the notorious over-smoothing problem, in which the learned representations converge to alike vectors when many layers are stacked. We propose Graph Highway Networks (GHNet) which utilize gating units to balance the trade-off between homogeneity and heterogeneity in the GCN learning process.
  arXiv  Detail & Related papers  (2020-04-09T16:26:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.