Robust Optimization as Data Augmentation for Large-scale Graphs
- URL: http://arxiv.org/abs/2010.09891v3
- Date: Tue, 29 Mar 2022 07:06:47 GMT
- Title: Robust Optimization as Data Augmentation for Large-scale Graphs
- Authors: Kezhi Kong, Guohao Li, Mucong Ding, Zuxuan Wu, Chen Zhu, Bernard
Ghanem, Gavin Taylor, Tom Goldstein
- Abstract summary: We propose FLAG (Free Large-scale Adversarial Augmentation on Graphs), which iteratively augments node features with gradient-based adversarial perturbations during training.
FLAG is a general-purpose approach for graph data, which universally works in node classification, link prediction, and graph classification tasks.
- Score: 117.2376815614148
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data augmentation helps neural networks generalize better by enlarging the
training set, but it remains an open question how to effectively augment graph
data to enhance the performance of GNNs (Graph Neural Networks). While most
existing graph regularizers focus on manipulating graph topological structures
by adding/removing edges, we offer a method to augment node features for better
performance. We propose FLAG (Free Large-scale Adversarial Augmentation on
Graphs), which iteratively augments node features with gradient-based
adversarial perturbations during training. By making the model invariant to
small fluctuations in input data, our method helps models generalize to
out-of-distribution samples and boosts model performance at test time. FLAG is
a general-purpose approach for graph data, which universally works in node
classification, link prediction, and graph classification tasks. FLAG is also
highly flexible and scalable, and is deployable with arbitrary GNN backbones
and large-scale datasets. We demonstrate the efficacy and stability of our
method through extensive experiments and ablation studies. We also provide
intuitive observations for a deeper understanding of our method.
Related papers
- Enhancing Size Generalization in Graph Neural Networks through Disentangled Representation Learning [7.448831299106425]
DISGEN is a model-agnostic framework designed to disentangle size factors from graph representations.
Our empirical results show that DISGEN outperforms the state-of-the-art models by up to 6% on real-world datasets.
arXiv Detail & Related papers (2024-06-07T03:19:24Z) - ENGAGE: Explanation Guided Data Augmentation for Graph Representation
Learning [34.23920789327245]
We propose ENGAGE, where explanation guides the contrastive augmentation process to preserve the key parts in graphs.
We also design two data augmentation schemes on graphs for perturbing structural and feature information, respectively.
arXiv Detail & Related papers (2023-07-03T14:33:14Z) - Graph Property Prediction on Open Graph Benchmark: A Winning Solution by
Graph Neural Architecture Search [37.89305885538052]
We design a graph neural network framework for graph classification task by introducing PAS(Pooling Architecture Search)
We improve it based on the GNN topology design method F2GNN to further improve the performance of the model in the graph property prediction task.
It is proved that the NAS method has high generalization ability for multiple tasks and the advantage of our method in processing graph property prediction tasks.
arXiv Detail & Related papers (2022-07-13T08:17:48Z) - Optimal Propagation for Graph Neural Networks [51.08426265813481]
We propose a bi-level optimization approach for learning the optimal graph structure.
We also explore a low-rank approximation model for further reducing the time complexity.
arXiv Detail & Related papers (2022-05-06T03:37:00Z) - Node Feature Extraction by Self-Supervised Multi-scale Neighborhood
Prediction [123.20238648121445]
We propose a new self-supervised learning framework, Graph Information Aided Node feature exTraction (GIANT)
GIANT makes use of the eXtreme Multi-label Classification (XMC) formalism, which is crucial for fine-tuning the language model based on graph information.
We demonstrate the superior performance of GIANT over the standard GNN pipeline on Open Graph Benchmark datasets.
arXiv Detail & Related papers (2021-10-29T19:55:12Z) - Local Augmentation for Graph Neural Networks [78.48812244668017]
We introduce the local augmentation, which enhances node features by its local subgraph structures.
Based on the local augmentation, we further design a novel framework: LA-GNN, which can apply to any GNN models in a plug-and-play manner.
arXiv Detail & Related papers (2021-09-08T18:10:08Z) - Training Robust Graph Neural Networks with Topology Adaptive Edge
Dropping [116.26579152942162]
Graph neural networks (GNNs) are processing architectures that exploit graph structural information to model representations from network data.
Despite their success, GNNs suffer from sub-optimal generalization performance given limited training data.
This paper proposes Topology Adaptive Edge Dropping to improve generalization performance and learn robust GNN models.
arXiv Detail & Related papers (2021-06-05T13:20:36Z) - GraphMI: Extracting Private Graph Data from Graph Neural Networks [59.05178231559796]
We present textbfGraph textbfModel textbfInversion attack (GraphMI), which aims to extract private graph data of the training graph by inverting GNN.
Specifically, we propose a projected gradient module to tackle the discreteness of graph edges while preserving the sparsity and smoothness of graph features.
We design a graph auto-encoder module to efficiently exploit graph topology, node attributes, and target model parameters for edge inference.
arXiv Detail & Related papers (2021-06-05T07:07:52Z) - Graph Contrastive Learning with Adaptive Augmentation [23.37786673825192]
We propose a novel graph contrastive representation learning method with adaptive augmentation.
Specifically, we design augmentation schemes based on node centrality measures to highlight important connective structures.
Our proposed method consistently outperforms existing state-of-the-art baselines and even surpasses some supervised counterparts.
arXiv Detail & Related papers (2020-10-27T15:12:21Z) - Data Augmentation for Graph Neural Networks [32.24311481878144]
We study graph data augmentation for graph neural networks (GNNs) in the context of improving semi-supervised node-classification.
Our work shows that neural edge predictors can effectively encode class-homophilic structure to promote intra-class edges and demote inter-class edges in given graph structure.
Our main contribution introduces the GAug graph data augmentation framework, which leverages these insights to improve performance in GNN-based node classification via edge prediction.
arXiv Detail & Related papers (2020-06-11T21:17:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.