Enhancing Size Generalization in Graph Neural Networks through Disentangled Representation Learning

• URL: http://arxiv.org/abs/2406.04601v3
• Date: Tue, 11 Jun 2024 21:10:58 GMT
• Title: Enhancing Size Generalization in Graph Neural Networks through Disentangled Representation Learning
• Authors: Zheng Huang, Qihui Yang, Dawei Zhou, Yujun Yan,
• Abstract summary: DISGEN is a model-agnostic framework designed to disentangle size factors from graph representations. Our empirical results show that DISGEN outperforms the state-of-the-art models by up to 6% on real-world datasets.
• Score: 7.448831299106425
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Although most graph neural networks (GNNs) can operate on graphs of any size, their classification performance often declines on graphs larger than those encountered during training. Existing methods insufficiently address the removal of size information from graph representations, resulting in sub-optimal performance and reliance on backbone models. In response, we propose DISGEN, a novel and model-agnostic framework designed to disentangle size factors from graph representations. DISGEN employs size- and task-invariant augmentations and introduces a decoupling loss that minimizes shared information in hidden representations, with theoretical guarantees for its effectiveness. Our empirical results show that DISGEN outperforms the state-of-the-art models by up to 6% on real-world datasets, underscoring its effectiveness in enhancing the size generalizability of GNNs. Our codes are available at: https://github.com/GraphmindDartmouth/DISGEN.

Related papers

• Graph Unlearning with Efficient Partial Retraining [28.433619085748447]
  Graph Neural Networks (GNNs) have achieved remarkable success in various real-world applications. GNNs may be trained on undesirable graph data, which can degrade their performance and reliability. We propose GraphRevoker, a novel graph unlearning framework that better maintains the model utility of unlearnable GNNs.
  arXiv  Detail & Related papers  (2024-03-12T06:22:10Z)
• Loss-aware Curriculum Learning for Heterogeneous Graph Neural Networks [30.333265803394998]
  This paper investigates the application of curriculum learning techniques to improve the performance of Heterogeneous Graph Neural Networks (GNNs) To better classify the quality of the data, we design a loss-aware training schedule, named LTS, that measures the quality of every nodes of the data. Our findings demonstrate the efficacy of curriculum learning in enhancing HGNNs capabilities for analyzing complex graph-structured data.
  arXiv  Detail & Related papers  (2024-02-29T05:44:41Z)
• Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks [52.566735716983956]
  We propose a graph gradual pruning framework termed CGP to dynamically prune GNNs. Unlike LTH-based methods, the proposed CGP approach requires no re-training, which significantly reduces the computation costs. Our proposed strategy greatly improves both training and inference efficiency while matching or even exceeding the accuracy of existing methods.
  arXiv  Detail & Related papers  (2022-07-18T14:23:31Z)
• SizeShiftReg: a Regularization Method for Improving Size-Generalization in Graph Neural Networks [5.008597638379227]
  Graph neural networks (GNNs) have become the de facto model of choice for graph classification. We propose a regularization strategy that can be applied to any GNN to improve its generalization capabilities without requiring access to the test data. Our regularization is based on the idea of simulating a shift in the size of the training graphs using coarsening techniques.
  arXiv  Detail & Related papers  (2022-07-16T09:50:45Z)
• Distribution Preserving Graph Representation Learning [11.340722297341788]
  Graph neural network (GNN) is effective to model graphs for distributed representations of nodes and an entire graph. We propose Distribution Preserving GNN (DP-GNN) - a GNN framework that can improve the generalizability of expressive GNN models. We evaluate the proposed DP-GNN framework on multiple benchmark datasets for graph classification tasks.
  arXiv  Detail & Related papers  (2022-02-27T19:16:26Z)
• OOD-GNN: Out-of-Distribution Generalized Graph Neural Network [73.67049248445277]
  Graph neural networks (GNNs) have achieved impressive performance when testing and training graph data come from identical distribution. Existing GNNs lack out-of-distribution generalization abilities so that their performance substantially degrades when there exist distribution shifts between testing and training graph data. We propose an out-of-distribution generalized graph neural network (OOD-GNN) for achieving satisfactory performance on unseen testing graphs that have different distributions with training graphs.
  arXiv  Detail & Related papers  (2021-12-07T16:29:10Z)
• Learning to Drop: Robust Graph Neural Network via Topological Denoising [50.81722989898142]
  We propose PTDNet, a parameterized topological denoising network, to improve the robustness and generalization performance of Graph Neural Networks (GNNs) PTDNet prunes task-irrelevant edges by penalizing the number of edges in the sparsified graph with parameterized networks. We show that PTDNet can improve the performance of GNNs significantly and the performance gain becomes larger for more noisy datasets.
  arXiv  Detail & Related papers  (2020-11-13T18:53:21Z)
• Robust Optimization as Data Augmentation for Large-scale Graphs [117.2376815614148]
  We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training. FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks.
  arXiv  Detail & Related papers  (2020-10-19T21:51:47Z)
• From Local Structures to Size Generalization in Graph Neural Networks [53.3202754533658]
  Graph neural networks (GNNs) can process graphs of different sizes. Their ability to generalize across sizes, specifically from small to large graphs, is still not well understood.
  arXiv  Detail & Related papers  (2020-10-17T19:36:54Z)
• Efficient Robustness Certificates for Discrete Data: Sparsity-Aware Randomized Smoothing for Graphs, Images and More [85.52940587312256]
  We propose a model-agnostic certificate based on the randomized smoothing framework which subsumes earlier work and is tight, efficient, and sparsity-aware. We show the effectiveness of our approach on a wide variety of models, datasets, and tasks -- specifically highlighting its use for Graph Neural Networks.
  arXiv  Detail & Related papers  (2020-08-29T10:09:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.