Elastic Graph Neural Networks

• URL: http://arxiv.org/abs/2107.06996v1
• Date: Mon, 5 Jul 2021 01:36:01 GMT
• Title: Elastic Graph Neural Networks
• Authors: Xiaorui Liu, Wei Jin, Yao Ma, Yaxin Li, Hua Liu, Yiqi Wang, Ming Yan, Jiliang Tang
• Abstract summary: We introduce a family of GNNs (Elastic GNNs) based on $ell$ and $ell$-based graph smoothing. In particular, we propose a novel and general message passing scheme into GNNs. Experiments on semi-supervised learning tasks demonstrate that the proposed Elastic GNNs obtain better adaptivity on benchmark datasets.
• Score: 32.805937635335255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While many existing graph neural networks (GNNs) have been proven to perform $\ell_2$-based graph smoothing that enforces smoothness globally, in this work we aim to further enhance the local smoothness adaptivity of GNNs via $\ell_1$-based graph smoothing. As a result, we introduce a family of GNNs (Elastic GNNs) based on $\ell_1$ and $\ell_2$-based graph smoothing. In particular, we propose a novel and general message passing scheme into GNNs. This message passing algorithm is not only friendly to back-propagation training but also achieves the desired smoothing properties with a theoretical convergence guarantee. Experiments on semi-supervised learning tasks demonstrate that the proposed Elastic GNNs obtain better adaptivity on benchmark datasets and are significantly robust to graph adversarial attacks. The implementation of Elastic GNNs is available at \url{https://github.com/lxiaorui/ElasticGNN}.

Related papers

• Spatio-Spectral Graph Neural Networks [50.277959544420455]
  We propose Spatio-Spectral Graph Networks (S$2$GNNs) S$2$GNNs combine spatially and spectrally parametrized graph filters. We show that S$2$GNNs vanquish over-squashing and yield strictly tighter approximation-theoretic error bounds than MPGNNs.
  arXiv  Detail & Related papers  (2024-05-29T14:28:08Z)
• Classifying Nodes in Graphs without GNNs [50.311528896010785]
  We propose a fully GNN-free approach for node classification, not requiring them at train or test time. Our method consists of three key components: smoothness constraints, pseudo-labeling iterations and neighborhood-label histograms.
  arXiv  Detail & Related papers  (2024-02-08T18:59:30Z)
• Learning to Reweight for Graph Neural Network [63.978102332612906]
  Graph Neural Networks (GNNs) show promising results for graph tasks. Existing GNNs' generalization ability will degrade when there exist distribution shifts between testing and training graph data. We propose a novel nonlinear graph decorrelation method, which can substantially improve the out-of-distribution generalization ability.
  arXiv  Detail & Related papers  (2023-12-19T12:25:10Z)
• Robust Graph Neural Networks using Weighted Graph Laplacian [1.8292714902548342]
  Graph neural network (GNN) is vulnerable to noise and adversarial attacks in input data. We propose a generic framework for robustifying GNN known as Weighted Laplacian GNN (RWL-GNN)
  arXiv  Detail & Related papers  (2022-08-03T05:36:35Z)
• $p$-Laplacian Based Graph Neural Networks [27.747195341003263]
  Graph networks (GNNs) have demonstrated superior performance for semi-supervised node classification on graphs. We propose a new $p$-Laplacian based GNN model, termed as $p$GNN, whose message passing mechanism is derived from a discrete regularization framework. We show that the new message passing mechanism works simultaneously as low-pass and high-pass filters, thus making $p$GNNs effective on both homophilic and heterophilic graphs.
  arXiv  Detail & Related papers  (2021-11-14T13:16:28Z)
• Graph Neural Networks with Local Graph Parameters [1.8600631687568656]
  Local graph parameters can be added to any Graph Neural Networks (GNNs) architecture. Our results connect GNNs with deep results in finite model theory and finite variable logics.
  arXiv  Detail & Related papers  (2021-06-12T07:43:51Z)
• Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth [57.10183643449905]
  Graph Neural Networks (GNNs) have been studied from the lens of expressive power and generalization. We study the dynamics of GNNs by studying deep skip optimization. Our results provide first theoretical support for the success of GNNs.
  arXiv  Detail & Related papers  (2021-05-10T17:59:01Z)
• A Unified Lottery Ticket Hypothesis for Graph Neural Networks [82.31087406264437]
  We present a unified GNN sparsification (UGS) framework that simultaneously prunes the graph adjacency matrix and the model weights. We further generalize the popular lottery ticket hypothesis to GNNs for the first time, by defining a graph lottery ticket (GLT) as a pair of core sub-dataset and sparse sub-network.
  arXiv  Detail & Related papers  (2021-02-12T21:52:43Z)
• Rethinking Graph Regularization for Graph Neural Networks [21.32758655943999]
  We show that graph Laplacian regularization brings little-to-no benefit to existing graph neural networks (GNNs) We propose a simple but non-trivial variant of graph Laplacian regularization, called propagation-regularization (P-reg) We demonstrate that P-reg can effectively boost the performance of existing GNN models on both node-level and graph-level tasks.
  arXiv  Detail & Related papers  (2020-09-04T07:04:51Z)
• GPT-GNN: Generative Pre-Training of Graph Neural Networks [93.35945182085948]
  Graph neural networks (GNNs) have been demonstrated to be powerful in modeling graph-structured data. We present the GPT-GNN framework to initialize GNNs by generative pre-training. We show that GPT-GNN significantly outperforms state-of-the-art GNN models without pre-training by up to 9.1% across various downstream tasks.
  arXiv  Detail & Related papers  (2020-06-27T20:12:33Z)
• Graph Random Neural Network for Semi-Supervised Learning on Graphs [36.218650686748546]
  We study the problem of semi-supervised learning on graphs, for which graph neural networks (GNNs) have been extensively explored. Most existing GNNs inherently suffer from the limitations of over-smoothing, non-robustness, and weak-generalization when labeled nodes are scarce. In this paper, we propose a simple yet effective framework -- GRAPH R NEURAL NETWORKS (GRAND) -- to address these issues.
  arXiv  Detail & Related papers  (2020-05-22T09:40:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.