Revisiting Graph Convolutional Network on Semi-Supervised Node
Classification from an Optimization Perspective
- URL: http://arxiv.org/abs/2009.11469v2
- Date: Fri, 25 Sep 2020 02:00:16 GMT
- Title: Revisiting Graph Convolutional Network on Semi-Supervised Node
Classification from an Optimization Perspective
- Authors: Hongwei Zhang, Tijin Yan, Zenjun Xie, Yuanqing Xia, Yuan Zhang
- Abstract summary: Graph convolutional networks (GCNs) have achieved promising performance on various graph-based tasks.
However they suffer from over-smoothing when stacking more layers.
We present a quantitative study on this observation and develop novel insights towards the deeper GCN.
- Score: 10.178145000390671
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Graph convolutional networks (GCNs) have achieved promising performance on
various graph-based tasks. However they suffer from over-smoothing when
stacking more layers. In this paper, we present a quantitative study on this
observation and develop novel insights towards the deeper GCN. First, we
interpret the current graph convolutional operations from an optimization
perspective and argue that over-smoothing is mainly caused by the naive
first-order approximation of the solution to the optimization problem.
Subsequently, we introduce two metrics to measure the over-smoothing on
node-level tasks. Specifically, we calculate the fraction of the pairwise
distance between connected and disconnected nodes to the overall distance
respectively. Based on our theoretical and empirical analysis, we establish a
universal theoretical framework of GCN from an optimization perspective and
derive a novel convolutional kernel named GCN+ which has lower parameter amount
while relieving the over-smoothing inherently. Extensive experiments on
real-world datasets demonstrate the superior performance of GCN+ over
state-of-the-art baseline methods on the node classification tasks.
Related papers
- Scalable Graph Compressed Convolutions [68.85227170390864]
We propose a differentiable method that applies permutations to calibrate input graphs for Euclidean convolution.
Based on the graph calibration, we propose the Compressed Convolution Network (CoCN) for hierarchical graph representation learning.
arXiv Detail & Related papers (2024-07-26T03:14:13Z) - From Spectral Graph Convolutions to Large Scale Graph Convolutional
Networks [0.0]
Graph Convolutional Networks (GCNs) have been shown to be a powerful concept that has been successfully applied to a large variety of tasks.
We study the theory that paved the way to the definition of GCN, including related parts of classical graph theory.
arXiv Detail & Related papers (2022-07-12T16:57:08Z) - ResNorm: Tackling Long-tailed Degree Distribution Issue in Graph Neural
Networks via Normalization [80.90206641975375]
This paper focuses on improving the performance of GNNs via normalization.
By studying the long-tailed distribution of node degrees in the graph, we propose a novel normalization method for GNNs.
The $scale$ operation of ResNorm reshapes the node-wise standard deviation (NStd) distribution so as to improve the accuracy of tail nodes.
arXiv Detail & Related papers (2022-06-16T13:49:09Z) - SStaGCN: Simplified stacking based graph convolutional networks [2.556756699768804]
Graph convolutional network (GCN) is a powerful model studied broadly in various graph structural data learning tasks.
We propose a novel GCN called SStaGCN (Simplified stacking based GCN) by utilizing the ideas of stacking and aggregation.
We show that SStaGCN can efficiently mitigate the over-smoothing problem of GCN.
arXiv Detail & Related papers (2021-11-16T05:00:08Z) - Tackling Oversmoothing of GNNs with Contrastive Learning [35.88575306925201]
Graph neural networks (GNNs) integrate the comprehensive relation of graph data and representation learning capability.
Oversmoothing makes the final representations of nodes indiscriminative, thus deteriorating the node classification and link prediction performance.
We propose the Topology-guided Graph Contrastive Layer, named TGCL, which is the first de-oversmoothing method maintaining all three mentioned metrics.
arXiv Detail & Related papers (2021-10-26T15:56:16Z) - Graph Partner Neural Networks for Semi-Supervised Learning on Graphs [16.489177915147785]
Graph Convolutional Networks (GCNs) are powerful for processing graphstructured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification.
It is inevitable for deep GCNs to suffer from an over-smoothing issue that the representations of nodes will tend to be indistinguishable after repeated graph convolution operations.
We propose the Graph Partner Neural Network (GPNN) which incorporates a de- parameterized GCN and a parameter-sharing scheme.
arXiv Detail & Related papers (2021-10-18T10:56:56Z) - Towards Efficient Graph Convolutional Networks for Point Cloud Handling [181.59146413326056]
We aim at improving the computational efficiency of graph convolutional networks (GCNs) for learning on point clouds.
A series of experiments show that optimized networks have reduced computational complexity, decreased memory consumption, and accelerated inference speed.
arXiv Detail & Related papers (2021-04-12T17:59:16Z) - Towards Deeper Graph Neural Networks [63.46470695525957]
Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations.
Several recent studies attribute this performance deterioration to the over-smoothing issue.
We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
arXiv Detail & Related papers (2020-07-18T01:11:14Z) - Infinitely Wide Graph Convolutional Networks: Semi-supervised Learning
via Gaussian Processes [144.6048446370369]
Graph convolutional neural networks(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification.
We propose a GP regression model via GCNs(GPGC) for graph-based semi-supervised learning.
We conduct extensive experiments to evaluate GPGC and demonstrate that it outperforms other state-of-the-art methods.
arXiv Detail & Related papers (2020-02-26T10:02:32Z) - Unifying Graph Convolutional Neural Networks and Label Propagation [73.82013612939507]
We study the relationship between LPA and GCN in terms of two aspects: feature/label smoothing and feature/label influence.
Based on our theoretical analysis, we propose an end-to-end model that unifies GCN and LPA for node classification.
Our model can also be seen as learning attention weights based on node labels, which is more task-oriented than existing feature-based attention models.
arXiv Detail & Related papers (2020-02-17T03:23:13Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.