GRAND+: Scalable Graph Random Neural Networks
- URL: http://arxiv.org/abs/2203.06389v1
- Date: Sat, 12 Mar 2022 09:41:23 GMT
- Title: GRAND+: Scalable Graph Random Neural Networks
- Authors: Wenzheng Feng, Yuxiao Dong, Tinglin Huang, Ziqi Yin, Xu Cheng, Evgeny
Kharlamov, Jie Tang
- Abstract summary: Graph neural networks (GNNs) have been widely adopted for semi-supervised learning on graphs.
It is difficult for GRAND to handle large-scale graphs since its effectiveness relies on computationally expensive data augmentation procedures.
We present a scalable and high-performance GNN framework GRAND+ for semi-supervised graph learning.
- Score: 26.47857017550499
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Graph neural networks (GNNs) have been widely adopted for semi-supervised
learning on graphs. A recent study shows that the graph random neural network
(GRAND) model can generate state-of-the-art performance for this problem.
However, it is difficult for GRAND to handle large-scale graphs since its
effectiveness relies on computationally expensive data augmentation procedures.
In this work, we present a scalable and high-performance GNN framework GRAND+
for semi-supervised graph learning. To address the above issue, we develop a
generalized forward push (GFPush) algorithm in GRAND+ to pre-compute a general
propagation matrix and employ it to perform graph data augmentation in a
mini-batch manner. We show that both the low time and space complexities of
GFPush enable GRAND+ to efficiently scale to large graphs. Furthermore, we
introduce a confidence-aware consistency loss into the model optimization of
GRAND+, facilitating GRAND+'s generalization superiority. We conduct extensive
experiments on seven public datasets of different sizes. The results
demonstrate that GRAND+ 1) is able to scale to large graphs and costs less
running time than existing scalable GNNs, and 2) can offer consistent accuracy
improvements over both full-batch and scalable GNNs across all datasets.
Related papers
- Graph Classification via Reference Distribution Learning: Theory and Practice [24.74871206083017]
This work introduces Graph Reference Distribution Learning (GRDL), an efficient and accurate graph classification method.
GRDL treats each graph's latent node embeddings given by GNN layers as a discrete distribution, enabling direct classification without global pooling.
Experiments on moderate-scale and large-scale graph datasets show the superiority of GRDL over the state-of-the-art.
arXiv Detail & Related papers (2024-08-21T06:42:22Z) - What Can We Learn from State Space Models for Machine Learning on Graphs? [11.38076877943004]
We propose Graph State Space Convolution (GSSC) as a principled extension of State Space Models (SSMs) to graph-structured data.
By leveraging global permutation-equivariant set aggregation and factorizable graph kernels, GSSC preserves all three advantages of SSMs.
Our findings highlight the potential of GSSC as a powerful and scalable model for graph machine learning.
arXiv Detail & Related papers (2024-06-09T15:03:36Z) - Training Graph Neural Networks on Growing Stochastic Graphs [114.75710379125412]
Graph Neural Networks (GNNs) rely on graph convolutions to exploit meaningful patterns in networked data.
We propose to learn GNNs on very large graphs by leveraging the limit object of a sequence of growing graphs, the graphon.
arXiv Detail & Related papers (2022-10-27T16:00:45Z) - A Comprehensive Study on Large-Scale Graph Training: Benchmarking and
Rethinking [124.21408098724551]
Large-scale graph training is a notoriously challenging problem for graph neural networks (GNNs)
We present a new ensembling training manner, named EnGCN, to address the existing issues.
Our proposed method has achieved new state-of-the-art (SOTA) performance on large-scale datasets.
arXiv Detail & Related papers (2022-10-14T03:43:05Z) - Graph Generative Model for Benchmarking Graph Neural Networks [73.11514658000547]
We introduce a novel graph generative model that learns and reproduces the distribution of real-world graphs in a privacy-controlled way.
Our model can successfully generate privacy-controlled, synthetic substitutes of large-scale real-world graphs that can be effectively used to benchmark GNN models.
arXiv Detail & Related papers (2022-07-10T06:42:02Z) - A Unified Lottery Ticket Hypothesis for Graph Neural Networks [82.31087406264437]
We present a unified GNN sparsification (UGS) framework that simultaneously prunes the graph adjacency matrix and the model weights.
We further generalize the popular lottery ticket hypothesis to GNNs for the first time, by defining a graph lottery ticket (GLT) as a pair of core sub-dataset and sparse sub-network.
arXiv Detail & Related papers (2021-02-12T21:52:43Z) - Distance Encoding: Design Provably More Powerful Neural Networks for
Graph Representation Learning [63.97983530843762]
Graph Neural Networks (GNNs) have achieved great success in graph representation learning.
GNNs generate identical representations for graph substructures that may in fact be very different.
More powerful GNNs, proposed recently by mimicking higher-order tests, are inefficient as they cannot sparsity of underlying graph structure.
We propose Distance Depiction (DE) as a new class of graph representation learning.
arXiv Detail & Related papers (2020-08-31T23:15:40Z) - Scaling Graph Neural Networks with Approximate PageRank [64.92311737049054]
We present the PPRGo model which utilizes an efficient approximation of information diffusion in GNNs.
In addition to being faster, PPRGo is inherently scalable, and can be trivially parallelized for large datasets like those found in industry settings.
We show that training PPRGo and predicting labels for all nodes in this graph takes under 2 minutes on a single machine, far outpacing other baselines on the same graph.
arXiv Detail & Related papers (2020-07-03T09:30:07Z) - Graph Random Neural Network for Semi-Supervised Learning on Graphs [36.218650686748546]
We study the problem of semi-supervised learning on graphs, for which graph neural networks (GNNs) have been extensively explored.
Most existing GNNs inherently suffer from the limitations of over-smoothing, non-robustness, and weak-generalization when labeled nodes are scarce.
In this paper, we propose a simple yet effective framework -- GRAPH R NEURAL NETWORKS (GRAND) -- to address these issues.
arXiv Detail & Related papers (2020-05-22T09:40:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.