Simple and Deep Graph Convolutional Networks
- URL: http://arxiv.org/abs/2007.02133v1
- Date: Sat, 4 Jul 2020 16:18:06 GMT
- Title: Simple and Deep Graph Convolutional Networks
- Authors: Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, Yaliang Li
- Abstract summary: Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data.
Despite their success, most of the current GCN models are shallow, due to the em over-smoothing problem.
We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques.
- Score: 63.76221532439285
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Graph convolutional networks (GCNs) are a powerful deep learning approach for
graph-structured data. Recently, GCNs and subsequent variants have shown
superior performance in various application areas on real-world datasets.
Despite their success, most of the current GCN models are shallow, due to the
{\em over-smoothing} problem. In this paper, we study the problem of designing
and analyzing deep graph convolutional networks. We propose the GCNII, an
extension of the vanilla GCN model with two simple yet effective techniques:
{\em Initial residual} and {\em Identity mapping}. We provide theoretical and
empirical evidence that the two techniques effectively relieves the problem of
over-smoothing. Our experiments show that the deep GCNII model outperforms the
state-of-the-art methods on various semi- and full-supervised tasks. Code is
available at https://github.com/chennnM/GCNII .
Related papers
- DRGCN: Dynamic Evolving Initial Residual for Deep Graph Convolutional
Networks [19.483662490506646]
We propose a novel model called Dynamic evolving initial Residual Graph Convolutional Network (DRGCN)
Our experimental results show that our model effectively relieves the problem of over-smoothing in deep GCNs.
Our model reaches new SOTA results on the large-scale ogbn-arxiv dataset of Open Graph Benchmark (OGB)
arXiv Detail & Related papers (2023-02-10T06:57:12Z) - Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural
Networks [52.566735716983956]
We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs.
Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs.
Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
arXiv Detail & Related papers (2022-07-18T14:23:31Z) - Multi-scale Graph Convolutional Networks with Self-Attention [2.66512000865131]
Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data.
Over-smoothing phenomenon as a crucial issue of GCNs remains to be solved and investigated.
We propose two novel multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs.
arXiv Detail & Related papers (2021-12-04T04:41:24Z) - SStaGCN: Simplified stacking based graph convolutional networks [2.556756699768804]
Graph convolutional network (GCN) is a powerful model studied broadly in various graph structural data learning tasks.
We propose a novel GCN called SStaGCN (Simplified stacking based GCN) by utilizing the ideas of stacking and aggregation.
We show that SStaGCN can efficiently mitigate the over-smoothing problem of GCN.
arXiv Detail & Related papers (2021-11-16T05:00:08Z) - Spatio-Temporal Inception Graph Convolutional Networks for
Skeleton-Based Action Recognition [126.51241919472356]
We design a simple and highly modularized graph convolutional network architecture for skeleton-based action recognition.
Our network is constructed by repeating a building block that aggregates multi-granularity information from both the spatial and temporal paths.
arXiv Detail & Related papers (2020-11-26T14:43:04Z) - Bi-GCN: Binary Graph Convolutional Network [57.733849700089955]
We propose a Binary Graph Convolutional Network (Bi-GCN), which binarizes both the network parameters and input node features.
Our Bi-GCN can reduce the memory consumption by an average of 30x for both the network parameters and input data, and accelerate the inference speed by an average of 47x.
arXiv Detail & Related papers (2020-10-15T07:26:23Z) - DeeperGCN: All You Need to Train Deeper GCNs [66.64739331859226]
Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs.
Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper.
This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs.
arXiv Detail & Related papers (2020-06-13T23:00:22Z) - Directed Graph Convolutional Network [15.879411956536885]
We extend spectral-based graph convolution to directed graphs by using first- and second-order proximity.
A new GCN model, called DGCN, is then designed to learn representations on the directed graph.
arXiv Detail & Related papers (2020-04-29T06:19:10Z) - Cross-GCN: Enhancing Graph Convolutional Network with $k$-Order Feature
Interactions [153.6357310444093]
Graph Convolutional Network (GCN) is an emerging technique that performs learning and reasoning on graph data.
We argue that existing designs of GCN forgo modeling cross features, making GCN less effective for tasks or data where cross features are important.
We design a new operator named Cross-feature Graph Convolution, which explicitly models the arbitrary-order cross features with complexity linear to feature dimension and order size.
arXiv Detail & Related papers (2020-03-05T13:05:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.