GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design

• URL: http://arxiv.org/abs/2112.11594v1
• Date: Wed, 22 Dec 2021 00:30:50 GMT
• Title: GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design
• Authors: Haoran You, Tong Geng, Yongan Zhang, Ang Li, Yingyan Lin
• Abstract summary: Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art graph learning model. It can be notoriously challenging to inference GCNs over large graph datasets. This paper proposes a GCN algorithm and accelerator Co-Design framework dubbed GCoD which can largely alleviate the aforementioned GCN irregularity.
• Score: 27.311994997480745
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Graph Convolutional Networks (GCNs) have emerged as the state-of-the-art graph learning model. However, it can be notoriously challenging to inference GCNs over large graph datasets, limiting their application to large real-world graphs and hindering the exploration of deeper and more sophisticated GCN graphs. This is because real-world graphs can be extremely large and sparse. Furthermore, the node degree of GCNs tends to follow the power-law distribution and therefore have highly irregular adjacency matrices, resulting in prohibitive inefficiencies in both data processing and movement and thus substantially limiting the achievable GCN acceleration efficiency. To this end, this paper proposes a GCN algorithm and accelerator Co-Design framework dubbed GCoD which can largely alleviate the aforementioned GCN irregularity and boost GCNs' inference efficiency. Specifically, on the algorithm level, GCoD integrates a split and conquer GCN training strategy that polarizes the graphs to be either denser or sparser in local neighborhoods without compromising the model accuracy, resulting in graph adjacency matrices that (mostly) have merely two levels of workload and enjoys largely enhanced regularity and thus ease of acceleration. On the hardware level, we further develop a dedicated two-pronged accelerator with a separated engine to process each of the aforementioned denser and sparser workloads, further boosting the overall utilization and acceleration efficiency. Extensive experiments and ablation studies validate that our GCoD consistently reduces the number of off-chip accesses, leading to speedups of 15286x, 294x, 7.8x, and 2.5x as compared to CPUs, GPUs, and prior-art GCN accelerators including HyGCN and AWB-GCN, respectively, while maintaining or even improving the task accuracy.

Related papers

• Graph Transformers for Large Graphs [57.19338459218758]
  This work advances representation learning on single large-scale graphs with a focus on identifying model characteristics and critical design constraints. A key innovation of this work lies in the creation of a fast neighborhood sampling technique coupled with a local attention mechanism. We report a 3x speedup and 16.8% performance gain on ogbn-products and snap-patents, while we also scale LargeGT on ogbn-100M with a 5.9% performance improvement.
  arXiv  Detail & Related papers  (2023-12-18T11:19:23Z)
• T-GAE: Transferable Graph Autoencoder for Network Alignment [79.89704126746204]
  T-GAE is a graph autoencoder framework that leverages transferability and stability of GNNs to achieve efficient network alignment without retraining. Our experiments demonstrate that T-GAE outperforms the state-of-the-art optimization method and the best GNN approach by up to 38.7% and 50.8%, respectively.
  arXiv  Detail & Related papers  (2023-10-05T02:58:29Z)
• SGCN: Exploiting Compressed-Sparse Features in Deep Graph Convolutional Network Accelerators [6.582242235154822]
  Graph convolutional networks (GCNs) are becoming increasingly popular as they overcome the limited applicability of prior neural networks. In this paper, we propose SGCN, a fast and energy-efficient GCN accelerator. We show that SGCN achieves 1.71x speedup and 43.9% higher energy efficiency compared to the existing accelerators.
  arXiv  Detail & Related papers  (2023-01-25T02:34:01Z)
• Beyond Graph Convolutional Network: An Interpretable Regularizer-centered Optimization Framework [12.116373546916078]
  Graph convolutional networks (GCNs) have been attracting widespread attentions due to their encouraging performance and powerful generalizations. In this paper, we induce an interpretable regularizer-centerd optimization framework, in which by building appropriate regularizers we can interpret most GCNs. Under the proposed framework, we devise a dual-regularizer graph convolutional network (dubbed tsGCN) to capture topological and semantic structures from graph data.
  arXiv  Detail & Related papers  (2023-01-11T05:51:33Z)
• H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture [13.149863422504332]
  H-GCN partitions each graph into three subgraphs based on its inherent heterogeneity, and processes them using PL and AIE, respectively. Compared with state-of-the-art GNN accelerators, H-GCN achieves, on average, speedups of 1.12.3X.
  arXiv  Detail & Related papers  (2022-06-28T03:37:31Z)
• BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Boundary Node Sampling [25.32242812045678]
  We propose a simple yet effective method dubbed BNS-GCN that adopts random Boundary-Node-Sampling to enable efficient and scalable distributed GCN training. Experiments and ablation studies consistently validate the effectiveness of BNS-GCN, boosting the throughput by up to 16.2x and reducing the memory usage by up to 58%, while maintaining a full-graph accuracy.
  arXiv  Detail & Related papers  (2022-03-21T13:44:37Z)
• GROW: A Row-Stationary Sparse-Dense GEMM Accelerator for Memory-Efficient Graph Convolutional Neural Networks [4.669338722185048]
  A unique property of Graph convolutional neural networks (GCNs) is that its two primary execution stages, aggregation and combination, exhibit drastically different dataflows. We present GROW, a GCN accelerator based on Gustavson's algorithm to architect a row-wise product based sparse-dense GEMM accelerator.
  arXiv  Detail & Related papers  (2022-03-01T00:26:31Z)
• Bi-GCN: Binary Graph Convolutional Network [57.733849700089955]
  We propose a Binary Graph Convolutional Network (Bi-GCN), which binarizes both the network parameters and input node features. Our Bi-GCN can reduce the memory consumption by an average of 30x for both the network parameters and input data, and accelerate the inference speed by an average of 47x.
  arXiv  Detail & Related papers  (2020-10-15T07:26:23Z)
• DeeperGCN: All You Need to Train Deeper GCNs [66.64739331859226]
  Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs.
  arXiv  Detail & Related papers  (2020-06-13T23:00:22Z)
• Graph Highway Networks [77.38665506495553]
  Graph Convolution Networks (GCN) are widely used in learning graph representations due to their effectiveness and efficiency. They suffer from the notorious over-smoothing problem, in which the learned representations converge to alike vectors when many layers are stacked. We propose Graph Highway Networks (GHNet) which utilize gating units to balance the trade-off between homogeneity and heterogeneity in the GCN learning process.
  arXiv  Detail & Related papers  (2020-04-09T16:26:43Z)
• L$^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks [118.37805042816784]
  Graph convolution networks (GCN) are increasingly popular in many applications, yet remain notoriously hard to train over large graph datasets. We propose a novel efficient layer-wise training framework for GCN (L-GCN), that disentangles feature aggregation and feature transformation during training. Experiments show that L-GCN is faster than state-of-the-arts by at least an order of magnitude, with a consistent of memory usage not dependent on dataset size.
  arXiv  Detail & Related papers  (2020-03-30T16:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.