Generalizing Graph Neural Networks on Out-Of-Distribution Graphs
- URL: http://arxiv.org/abs/2111.10657v4
- Date: Sun, 10 Mar 2024 07:52:04 GMT
- Title: Generalizing Graph Neural Networks on Out-Of-Distribution Graphs
- Authors: Shaohua Fan, Xiao Wang, Chuan Shi, Peng Cui and Bai Wang
- Abstract summary: Graph Neural Networks (GNNs) are proposed without considering the distribution shifts between training and testing graphs.
In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation.
We propose a general causal representation framework, called StableGNN, to eliminate the impact of spurious correlations.
- Score: 51.33152272781324
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Graph Neural Networks (GNNs) are proposed without considering the agnostic
distribution shifts between training and testing graphs, inducing the
degeneration of the generalization ability of GNNs on Out-Of-Distribution (OOD)
settings. The fundamental reason for such degeneration is that most GNNs are
developed based on the I.I.D hypothesis. In such a setting, GNNs tend to
exploit subtle statistical correlations existing in the training set for
predictions, even though it is a spurious correlation. However, such spurious
correlations may change in testing environments, leading to the failure of
GNNs. Therefore, eliminating the impact of spurious correlations is crucial for
stable GNNs. To this end, we propose a general causal representation framework,
called StableGNN. The main idea is to extract high-level representations from
graph data first and resort to the distinguishing ability of causal inference
to help the model get rid of spurious correlations. Particularly, we exploit a
graph pooling layer to extract subgraph-based representations as high-level
representations. Furthermore, we propose a causal variable distinguishing
regularizer to correct the biased training distribution. Hence, GNNs would
concentrate more on the stable correlations. Extensive experiments on both
synthetic and real-world OOD graph datasets well verify the effectiveness,
flexibility and interpretability of the proposed framework.
Related papers
- Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts.
Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor.
Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
arXiv Detail & Related papers (2024-02-18T07:49:22Z) - Learning to Reweight for Graph Neural Network [63.978102332612906]
Graph Neural Networks (GNNs) show promising results for graph tasks.
Existing GNNs' generalization ability will degrade when there exist distribution shifts between testing and training graph data.
We propose a novel nonlinear graph decorrelation method, which can substantially improve the out-of-distribution generalization ability.
arXiv Detail & Related papers (2023-12-19T12:25:10Z) - Towards Robust Fidelity for Evaluating Explainability of Graph Neural Networks [32.345435955298825]
Graph Neural Networks (GNNs) are neural models that leverage the dependency structure in graphical data via message passing among the graph nodes.
A main challenge in studying GNN explainability is to provide fidelity measures that evaluate the performance of these explanation functions.
This paper studies this foundational challenge, spotlighting the inherent limitations of prevailing fidelity metrics.
arXiv Detail & Related papers (2023-10-03T06:25:14Z) - Debiasing Graph Neural Networks via Learning Disentangled Causal
Substructure [46.86463923605841]
We present a graph classification investigation on the training graphs with severe bias.
We discover that GNNs always tend to explore the spurious correlations to make decision.
We propose a general disentangled GNN framework to learn the causal substructure and bias substructure.
arXiv Detail & Related papers (2022-09-28T13:55:52Z) - MentorGNN: Deriving Curriculum for Pre-Training GNNs [61.97574489259085]
We propose an end-to-end model named MentorGNN that aims to supervise the pre-training process of GNNs across graphs.
We shed new light on the problem of domain adaption on relational data (i.e., graphs) by deriving a natural and interpretable upper bound on the generalization error of the pre-trained GNNs.
arXiv Detail & Related papers (2022-08-21T15:12:08Z) - Discovering Invariant Rationales for Graph Neural Networks [104.61908788639052]
Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features.
We propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs.
arXiv Detail & Related papers (2022-01-30T16:43:40Z) - OOD-GNN: Out-of-Distribution Generalized Graph Neural Network [73.67049248445277]
Graph neural networks (GNNs) have achieved impressive performance when testing and training graph data come from identical distribution.
Existing GNNs lack out-of-distribution generalization abilities so that their performance substantially degrades when there exist distribution shifts between testing and training graph data.
We propose an out-of-distribution generalized graph neural network (OOD-GNN) for achieving satisfactory performance on unseen testing graphs that have different distributions with training graphs.
arXiv Detail & Related papers (2021-12-07T16:29:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.