Improving Graph Neural Networks with Simple Architecture Design
- URL: http://arxiv.org/abs/2105.07634v1
- Date: Mon, 17 May 2021 06:46:01 GMT
- Title: Improving Graph Neural Networks with Simple Architecture Design
- Authors: Sunil Kumar Maurya, Xin Liu and Tsuyoshi Murata
- Abstract summary: We introduce several key design strategies for graph neural networks.
We present a simple and shallow model, Feature Selection Graph Neural Network (FSGNN)
We show that the proposed model outperforms other state of the art GNN models and achieves up to 64% improvements in accuracy on node classification tasks.
- Score: 7.057970273958933
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Graph Neural Networks have emerged as a useful tool to learn on the data by
applying additional constraints based on the graph structure. These graphs are
often created with assumed intrinsic relations between the entities. In recent
years, there have been tremendous improvements in the architecture design,
pushing the performance up in various prediction tasks. In general, these
neural architectures combine layer depth and node feature aggregation steps.
This makes it challenging to analyze the importance of features at various hops
and the expressiveness of the neural network layers. As different graph
datasets show varying levels of homophily and heterophily in features and class
label distribution, it becomes essential to understand which features are
important for the prediction tasks without any prior information. In this work,
we decouple the node feature aggregation step and depth of graph neural network
and introduce several key design strategies for graph neural networks. More
specifically, we propose to use softmax as a regularizer and "Soft-Selector" of
features aggregated from neighbors at different hop distances; and
"Hop-Normalization" over GNN layers. Combining these techniques, we present a
simple and shallow model, Feature Selection Graph Neural Network (FSGNN), and
show empirically that the proposed model outperforms other state of the art GNN
models and achieves up to 64% improvements in accuracy on node classification
tasks. Moreover, analyzing the learned soft-selection parameters of the model
provides a simple way to study the importance of features in the prediction
tasks. Finally, we demonstrate with experiments that the model is scalable for
large graphs with millions of nodes and billions of edges.
Related papers
- DA-MoE: Addressing Depth-Sensitivity in Graph-Level Analysis through Mixture of Experts [70.21017141742763]
Graph neural networks (GNNs) are gaining popularity for processing graph-structured data.
Existing methods generally use a fixed number of GNN layers to generate representations for all graphs.
We propose the depth adaptive mixture of expert (DA-MoE) method, which incorporates two main improvements to GNN.
arXiv Detail & Related papers (2024-11-05T11:46:27Z) - Hyperbolic Benchmarking Unveils Network Topology-Feature Relationship in GNN Performance [0.5416466085090772]
We introduce a comprehensive benchmarking framework for graph machine learning.
We generate synthetic networks with realistic topological properties and node feature vectors.
Results highlight the dependency of model performance on the interplay between network structure and node features.
arXiv Detail & Related papers (2024-06-04T20:40:06Z) - Spectral Greedy Coresets for Graph Neural Networks [61.24300262316091]
The ubiquity of large-scale graphs in node-classification tasks hinders the real-world applications of Graph Neural Networks (GNNs)
This paper studies graph coresets for GNNs and avoids the interdependence issue by selecting ego-graphs based on their spectral embeddings.
Our spectral greedy graph coreset (SGGC) scales to graphs with millions of nodes, obviates the need for model pre-training, and applies to low-homophily graphs.
arXiv Detail & Related papers (2024-05-27T17:52:12Z) - Graph Property Prediction on Open Graph Benchmark: A Winning Solution by
Graph Neural Architecture Search [37.89305885538052]
We design a graph neural network framework for graph classification task by introducing PAS(Pooling Architecture Search)
We improve it based on the GNN topology design method F2GNN to further improve the performance of the model in the graph property prediction task.
It is proved that the NAS method has high generalization ability for multiple tasks and the advantage of our method in processing graph property prediction tasks.
arXiv Detail & Related papers (2022-07-13T08:17:48Z) - A Robust Stacking Framework for Training Deep Graph Models with
Multifaceted Node Features [61.92791503017341]
Graph Neural Networks (GNNs) with numerical node features and graph structure as inputs have demonstrated superior performance on various supervised learning tasks with graph data.
The best models for such data types in most standard supervised learning settings with IID (non-graph) data are not easily incorporated into a GNN.
Here we propose a robust stacking framework that fuses graph-aware propagation with arbitrary models intended for IID data.
arXiv Detail & Related papers (2022-06-16T22:46:33Z) - Simplifying approach to Node Classification in Graph Neural Networks [7.057970273958933]
We decouple the node feature aggregation step and depth of graph neural network, and empirically analyze how different aggregated features play a role in prediction performance.
We show that not all features generated via aggregation steps are useful, and often using these less informative features can be detrimental to the performance of the GNN model.
We present a simple and shallow model, Feature Selection Graph Neural Network (FSGNN), and show empirically that the proposed model achieves comparable or even higher accuracy than state-of-the-art GNN models.
arXiv Detail & Related papers (2021-11-12T14:53:22Z) - Breaking the Limit of Graph Neural Networks by Improving the
Assortativity of Graphs with Local Mixing Patterns [19.346133577539394]
Graph neural networks (GNNs) have achieved tremendous success on multiple graph-based learning tasks.
We focus on transforming the input graph into a computation graph which contains both proximity and structural information.
We show that adaptively choosing between structure and proximity leads to improved performance under diverse mixing.
arXiv Detail & Related papers (2021-06-11T19:18:34Z) - A Unified View on Graph Neural Networks as Graph Signal Denoising [49.980783124401555]
Graph Neural Networks (GNNs) have risen to prominence in learning representations for graph structured data.
In this work, we establish mathematically that the aggregation processes in a group of representative GNN models can be regarded as solving a graph denoising problem.
We instantiate a novel GNN model, ADA-UGNN, derived from UGNN, to handle graphs with adaptive smoothness across nodes.
arXiv Detail & Related papers (2020-10-05T04:57:18Z) - Towards Deeper Graph Neural Networks [63.46470695525957]
Graph convolutions perform neighborhood aggregation and represent one of the most important graph operations.
Several recent studies attribute this performance deterioration to the over-smoothing issue.
We propose Deep Adaptive Graph Neural Network (DAGNN) to adaptively incorporate information from large receptive fields.
arXiv Detail & Related papers (2020-07-18T01:11:14Z) - Graph Neural Networks Including Sparse Interpretability [0.0]
We present a model-agnostic framework for interpreting important graph structure and node features.
Our GISST models achieve superior node feature and edge explanation precision in synthetic datasets.
arXiv Detail & Related papers (2020-06-30T21:35:55Z) - Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph
Neural Networks [183.97265247061847]
We leverage graph signal processing to characterize the representation space of graph neural networks (GNNs)
We discuss the role of graph convolutional filters in GNNs and show that any architecture built with such filters has the fundamental properties of permutation equivariance and stability to changes in the topology.
We also study the use of GNNs in recommender systems and learning decentralized controllers for robot swarms.
arXiv Detail & Related papers (2020-03-08T13:02:15Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.