Multi-scale Graph Convolutional Networks with Self-Attention

• URL: http://arxiv.org/abs/2112.03262v1
• Date: Sat, 4 Dec 2021 04:41:24 GMT
• Title: Multi-scale Graph Convolutional Networks with Self-Attention
• Authors: Zhilong Xiong, Jia Cai
• Abstract summary: Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data. Over-smoothing phenomenon as a crucial issue of GCNs remains to be solved and investigated. We propose two novel multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs.
• Score: 2.66512000865131
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data recently. In general, deep GCNs do not work well since graph convolution in conventional GCNs is a special form of Laplacian smoothing, which makes the representation of different nodes indistinguishable. In the literature, multi-scale information was employed in GCNs to enhance the expressive power of GCNs. However, over-smoothing phenomenon as a crucial issue of GCNs remains to be solved and investigated. In this paper, we propose two novel multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs. Our methods greatly improve the computational efficiency and prediction accuracy of the GCNs model. Extensive experiments on both node classification and graph classification demonstrate the effectiveness over several state-of-the-art GCNs. Notably, the proposed two architectures can efficiently mitigate the over-smoothing problem of GCNs, and the layer of our model can even be increased to $64$.

Related papers

• SStaGCN: Simplified stacking based graph convolutional networks [2.556756699768804]
  Graph convolutional network (GCN) is a powerful model studied broadly in various graph structural data learning tasks. We propose a novel GCN called SStaGCN (Simplified stacking based GCN) by utilizing the ideas of stacking and aggregation. We show that SStaGCN can efficiently mitigate the over-smoothing problem of GCN.
  arXiv  Detail & Related papers  (2021-11-16T05:00:08Z)
• Graph Partner Neural Networks for Semi-Supervised Learning on Graphs [16.489177915147785]
  Graph Convolutional Networks (GCNs) are powerful for processing graphstructured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification. It is inevitable for deep GCNs to suffer from an over-smoothing issue that the representations of nodes will tend to be indistinguishable after repeated graph convolution operations. We propose the Graph Partner Neural Network (GPNN) which incorporates a de- parameterized GCN and a parameter-sharing scheme.
  arXiv  Detail & Related papers  (2021-10-18T10:56:56Z)
• Tackling Over-Smoothing for General Graph Convolutional Networks [88.71154017107257]
  We study how general GCNs act with the increase in depth, including generic GCN, GCN with bias, ResGCN, and APPNP. We propose DropEdge to alleviate over-smoothing by randomly removing a certain number of edges at each training epoch.
  arXiv  Detail & Related papers  (2020-08-22T16:14:01Z)
• AM-GCN: Adaptive Multi-channel Graph Convolutional Networks [85.0332394224503]
  We study whether Graph Convolutional Networks (GCNs) can optimally integrate node features and topological structures in a complex graph with rich information. We propose an adaptive multi-channel graph convolutional networks for semi-supervised classification (AM-GCN) Our experiments show that AM-GCN extracts the most correlated information from both node features and topological structures substantially.
  arXiv  Detail & Related papers  (2020-07-05T08:16:03Z)
• Simple and Deep Graph Convolutional Networks [63.76221532439285]
  Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data. Despite their success, most of the current GCN models are shallow, due to the em over-smoothing problem. We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques.
  arXiv  Detail & Related papers  (2020-07-04T16:18:06Z)
• Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks [62.8504260693664]
  Graph Convolutional Networks (GCNs) show promising results for semisupervised learning tasks on graphs. In this paper, we analyze GCNs in regard to the node degree distribution. We develop a novel Self-Supervised DegreeSpecific GCN (SL-DSGC) that mitigates the degree biases of GCNs.
  arXiv  Detail & Related papers  (2020-06-28T16:26:47Z)
• DeeperGCN: All You Need to Train Deeper GCNs [66.64739331859226]
  Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs. Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper. This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs.
  arXiv  Detail & Related papers  (2020-06-13T23:00:22Z)
• Cross-GCN: Enhancing Graph Convolutional Network with $k$-Order Feature Interactions [153.6357310444093]
  Graph Convolutional Network (GCN) is an emerging technique that performs learning and reasoning on graph data. We argue that existing designs of GCN forgo modeling cross features, making GCN less effective for tasks or data where cross features are important. We design a new operator named Cross-feature Graph Convolution, which explicitly models the arbitrary-order cross features with complexity linear to feature dimension and order size.
  arXiv  Detail & Related papers  (2020-03-05T13:05:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.