DeeperGCN: All You Need to Train Deeper GCNs

• URL: http://arxiv.org/abs/2006.07739v1
• Date: Sat, 13 Jun 2020 23:00:22 GMT
• Title: DeeperGCN: All You Need to Train Deeper GCNs
• Authors: Guohao Li, Chenxin Xiong, Ali Thabet, Bernard Ghanem
• Abstract summary: Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs.
• Score: 66.64739331859226
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. These challenges limit the representation power of GCNs on large-scale graphs. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs. We define differentiable generalized aggregation functions to unify different message aggregation operations (e.g. mean, max). We also propose a novel normalization layer namely MsgNorm and a pre-activation version of residual connections for GCNs. Extensive experiments on Open Graph Benchmark (OGB) show DeeperGCN significantly boosts performance over the state-of-the-art on the large scale graph learning tasks of node property prediction and graph property prediction. Please visit https://www.deepgcns.org for more information.

Related papers

• Beyond Graph Convolutional Network: An Interpretable Regularizer-centered Optimization Framework [12.116373546916078]
  Graph convolutional networks (GCNs) have been attracting widespread attentions due to their encouraging performance and powerful generalizations. In this paper, we induce an interpretable regularizer-centerd optimization framework, in which by building appropriate regularizers we can interpret most GCNs. Under the proposed framework, we devise a dual-regularizer graph convolutional network (dubbed tsGCN) to capture topological and semantic structures from graph data.
  arXiv  Detail & Related papers  (2023-01-11T05:51:33Z)
• Multi-scale Graph Convolutional Networks with Self-Attention [2.66512000865131]
  Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data. Over-smoothing phenomenon as a crucial issue of GCNs remains to be solved and investigated. We propose two novel multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs.
  arXiv  Detail & Related papers  (2021-12-04T04:41:24Z)
• Graph Partner Neural Networks for Semi-Supervised Learning on Graphs [16.489177915147785]
  Graph Convolutional Networks (GCNs) are powerful for processing graphstructured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification. It is inevitable for deep GCNs to suffer from an over-smoothing issue that the representations of nodes will tend to be indistinguishable after repeated graph convolution operations. We propose the Graph Partner Neural Network (GPNN) which incorporates a de- parameterized GCN and a parameter-sharing scheme.
  arXiv  Detail & Related papers  (2021-10-18T10:56:56Z)
• Node Feature Kernels Increase Graph Convolutional Network Robustness [19.076912727990326]
  The robustness of Graph Convolutional Networks (GCNs) to perturbations of their input is becoming a topic of increasing importance. In this paper, a random matrix theory analysis is possible. It is observed that enhancing the message passing step in GCNs by adding the node feature kernel to the adjacency matrix of the graph structure solves this problem.
  arXiv  Detail & Related papers  (2021-09-04T04:20:45Z)
• Graph Convolutional Networks for Graphs Containing Missing Features [5.426650977249329]
  We propose an approach that adapts Graph Convolutional Network (GCN) to graphs containing missing features. In contrast to traditional strategy, our approach integrates the processing of missing features and graph learning within the same neural network architecture. We demonstrate through extensive experiments that our approach significantly outperforms the imputation-based methods in node classification and link prediction tasks.
  arXiv  Detail & Related papers  (2020-07-09T06:47:21Z)
• AM-GCN: Adaptive Multi-channel Graph Convolutional Networks [85.0332394224503]
  We study whether Graph Convolutional Networks (GCNs) can optimally integrate node features and topological structures in a complex graph with rich information. We propose an adaptive multi-channel graph convolutional networks for semi-supervised classification (AM-GCN) Our experiments show that AM-GCN extracts the most correlated information from both node features and topological structures substantially.
  arXiv  Detail & Related papers  (2020-07-05T08:16:03Z)
• Simple and Deep Graph Convolutional Networks [63.76221532439285]
  Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Despite their success, most of the current GCN models are shallow, due to the em over-smoothing problem. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques.
  arXiv  Detail & Related papers  (2020-07-04T16:18:06Z)
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks [62.8504260693664]
  Graph Convolutional Networks (GCNs) show promising results for semisupervised learning tasks on graphs. In this paper, we analyze GCNs in regard to the node degree distribution. We develop a novel Self-Supervised DegreeSpecific GCN (SL-DSGC) that mitigates the degree biases of GCNs.
  arXiv  Detail & Related papers  (2020-06-28T16:26:47Z)
• Understanding and Resolving Performance Degradation in Graph Convolutional Networks [105.14867349802898]
  Graph Convolutional Network (GCN) stacks several layers and in each layer performs a PROPagation operation (PROP) and a TRANsformation operation (TRAN) for learning node representations over graph-structured data. GCNs tend to suffer performance drop when the model gets deep. We study performance degradation of GCNs by experimentally examining how stacking only TRANs or PROPs works.
  arXiv  Detail & Related papers  (2020-06-12T12:12:12Z)
• L$^2$-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks [118.37805042816784]
  Graph convolution networks (GCN) are increasingly popular in many applications, yet remain notoriously hard to train over large graph datasets. We propose a novel efficient layer-wise training framework for GCN (L-GCN), that disentangles feature aggregation and feature transformation during training. Experiments show that L-GCN is faster than state-of-the-arts by at least an order of magnitude, with a consistent of memory usage not dependent on dataset size.
  arXiv  Detail & Related papers  (2020-03-30T16:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.