Automated Graph Learning via Population Based Self-Tuning GCN
- URL: http://arxiv.org/abs/2107.04713v1
- Date: Fri, 9 Jul 2021 23:05:21 GMT
- Title: Automated Graph Learning via Population Based Self-Tuning GCN
- Authors: Ronghang Zhu and Zhiqiang Tao and Yaliang Li and Sheng Li
- Abstract summary: Graph convolutional network (GCN) and its variants have been successfully applied to a broad range of tasks.
Traditional GCN models suffer from the issues of overfitting and oversmoothing.
Recent techniques like DropEdge could alleviate these issues and thus enable the development of deep GCN.
- Score: 45.28411311903644
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Owing to the remarkable capability of extracting effective graph embeddings,
graph convolutional network (GCN) and its variants have been successfully
applied to a broad range of tasks, such as node classification, link
prediction, and graph classification. Traditional GCN models suffer from the
issues of overfitting and oversmoothing, while some recent techniques like
DropEdge could alleviate these issues and thus enable the development of deep
GCN. However, training GCN models is non-trivial, as it is sensitive to the
choice of hyperparameters such as dropout rate and learning weight decay,
especially for deep GCN models. In this paper, we aim to automate the training
of GCN models through hyperparameter optimization. To be specific, we propose a
self-tuning GCN approach with an alternate training algorithm, and further
extend our approach by incorporating the population based training scheme.
Experimental results on three benchmark datasets demonstrate the effectiveness
of our approaches on optimizing multi-layer GCN, compared with several
representative baselines.
Related papers
- Attentional Graph Neural Networks for Robust Massive Network
Localization [20.416879207269446]
Graph neural networks (GNNs) have emerged as a prominent tool for classification tasks in machine learning.
This paper integrates GNNs with attention mechanism to tackle a challenging nonlinear regression problem: network localization.
We first introduce a novel network localization method based on graph convolutional network (GCN), which exhibits exceptional precision even under severe non-line-of-sight (NLOS) conditions.
arXiv Detail & Related papers (2023-11-28T15:05:13Z) - Label Deconvolution for Node Representation Learning on Large-scale
Attributed Graphs against Learning Bias [75.44877675117749]
We propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs.
Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph datasets Benchmark.
arXiv Detail & Related papers (2023-09-26T13:09:43Z) - Neighborhood Convolutional Network: A New Paradigm of Graph Neural
Networks for Node Classification [12.062421384484812]
Graph Convolutional Network (GCN) decouples neighborhood aggregation and feature transformation in each convolutional layer.
In this paper, we propose a new paradigm of GCN, termed Neighborhood Convolutional Network (NCN)
In this way, the model could inherit the merit of decoupled GCN for aggregating neighborhood information, at the same time, develop much more powerful feature learning modules.
arXiv Detail & Related papers (2022-11-15T02:02:51Z) - Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural
Networks [52.566735716983956]
We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs.
Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs.
Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
arXiv Detail & Related papers (2022-07-18T14:23:31Z) - Multi-scale Graph Convolutional Networks with Self-Attention [2.66512000865131]
Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data.
Over-smoothing phenomenon as a crucial issue of GCNs remains to be solved and investigated.
We propose two novel multi-scale GCN frameworks by incorporating self-attention mechanism and multi-scale information into the design of GCNs.
arXiv Detail & Related papers (2021-12-04T04:41:24Z) - SStaGCN: Simplified stacking based graph convolutional networks [2.556756699768804]
Graph convolutional network (GCN) is a powerful model studied broadly in various graph structural data learning tasks.
We propose a novel GCN called SStaGCN (Simplified stacking based GCN) by utilizing the ideas of stacking and aggregation.
We show that SStaGCN can efficiently mitigate the over-smoothing problem of GCN.
arXiv Detail & Related papers (2021-11-16T05:00:08Z) - AdaGCN:Adaptive Boosting Algorithm for Graph Convolutional Networks on
Imbalanced Node Classification [10.72543417177307]
We propose an ensemble model called AdaGCN, which uses a Graph Convolutional Network (GCN) as the base estimator during adaptive boosting.
Our model also improves state-of-the-art baselines on all of the challenging node classification tasks we consider.
arXiv Detail & Related papers (2021-05-25T02:43:31Z) - Simple and Deep Graph Convolutional Networks [63.76221532439285]
Graph convolutional networks (GCNs) are a powerful deep learning approach for graph-structured data.
Despite their success, most of the current GCN models are shallow, due to the em over-smoothing problem.
We propose the GCNII, an extension of the vanilla GCN model with two simple yet effective techniques.
arXiv Detail & Related papers (2020-07-04T16:18:06Z) - DeeperGCN: All You Need to Train Deeper GCNs [66.64739331859226]
Graph Convolutional Networks (GCNs) have been drawing significant attention with the power of representation learning on graphs.
Unlike Convolutional Neural Networks (CNNs), which are able to take advantage of stacking very deep layers, GCNs suffer from vanishing gradient, over-smoothing and over-fitting issues when going deeper.
This paper proposes DeeperGCN that is capable of successfully and reliably training very deep GCNs.
arXiv Detail & Related papers (2020-06-13T23:00:22Z) - Cross-GCN: Enhancing Graph Convolutional Network with $k$-Order Feature
Interactions [153.6357310444093]
Graph Convolutional Network (GCN) is an emerging technique that performs learning and reasoning on graph data.
We argue that existing designs of GCN forgo modeling cross features, making GCN less effective for tasks or data where cross features are important.
We design a new operator named Cross-feature Graph Convolution, which explicitly models the arbitrary-order cross features with complexity linear to feature dimension and order size.
arXiv Detail & Related papers (2020-03-05T13:05:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.