Handling Distribution Shifts on Graphs: An Invariance Perspective
- URL: http://arxiv.org/abs/2202.02466v1
- Date: Sat, 5 Feb 2022 02:31:01 GMT
- Title: Handling Distribution Shifts on Graphs: An Invariance Perspective
- Authors: Qitian Wu, Hengrui Zhang, Junchi Yan, David Wipf
- Abstract summary: We formulate the OOD problem for node-level prediction on graphs.
We develop a new domain-invariant learning approach, named Explore-to-Extrapolate Risk Minimization.
We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
- Score: 77.14319095965058
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: There is increasing evidence suggesting neural networks' sensitivity to
distribution shifts, so that research on out-of-distribution (OOD)
generalization comes into the spotlight. Nonetheless, current endeavors mostly
focus on Euclidean data, and its formulation for graph-structured data is not
clear and remains under-explored, given the two-fold fundamental challenges: 1)
the inter-connection among nodes in one graph, which induces non-IID generation
of data points even under the same environment, and 2) the structural
information in the input graph, which is also informative for prediction. In
this paper, we formulate the OOD problem for node-level prediction on graphs
and develop a new domain-invariant learning approach, named
Explore-to-Extrapolate Risk Minimization, that facilitates GNNs to leverage
invariant graph features for prediction. The key difference to existing
invariant models is that we design multiple context explorers (specified as
graph editers in our case) that are adversarially trained to maximize the
variance of risks from multiple virtual environments. Such a design enables the
model to extrapolate from a single observed environment which is the common
case for node-level prediction. We prove the validity of our method by
theoretically showing its guarantee of a valid OOD solution and further
demonstrate its power on various real-world datasets for handling distribution
shifts from artificial spurious features, cross-domain transfers and dynamic
graph evolution.
Related papers
- Out-of-Distribution Detection on Graphs: A Survey [58.47395497985277]
Graph out-of-distribution (GOOD) detection focuses on identifying graph data that deviates from the distribution seen during training.
We categorize existing methods into four types: enhancement-based, reconstruction-based, information propagation-based, and classification-based approaches.
We discuss practical applications and theoretical foundations, highlighting the unique challenges posed by graph data.
arXiv Detail & Related papers (2025-02-12T04:07:12Z) - Generative Risk Minimization for Out-of-Distribution Generalization on Graphs [71.48583448654522]
We propose an innovative framework, named Generative Risk Minimization (GRM), designed to generate an invariant subgraph for each input graph to be classified, instead of extraction.
We conduct extensive experiments across a variety of real-world graph datasets for both node-level and graph-level OOD generalization.
arXiv Detail & Related papers (2025-02-11T21:24:13Z) - diffIRM: A Diffusion-Augmented Invariant Risk Minimization Framework for Spatiotemporal Prediction over Graphs [6.677219861416146]
Intemporal prediction over graphs (GSTP) is challenging, because real-world data suffers from the Out-of-Distribution (OOD) problem.
In this study, we propose a diffusion-augmented invariant risk minimization (diffIRM) framework that combines these two principles.
arXiv Detail & Related papers (2024-12-31T06:45:47Z) - IENE: Identifying and Extrapolating the Node Environment for Out-of-Distribution Generalization on Graphs [10.087216264788097]
We propose IENE, an OOD generalization method on graphs based on node-level environmental identification and extrapolation techniques.
It strengthens the model's ability to extract invariance from two granularities simultaneously, leading to improved generalization.
arXiv Detail & Related papers (2024-06-02T14:43:56Z) - Graphs Generalization under Distribution Shifts [11.963958151023732]
We introduce a novel framework, namely Graph Learning Invariant Domain genERation (GLIDER)
Our model outperforms baseline methods on node-level OOD generalization across domains in distribution shift on node features and topological structures simultaneously.
arXiv Detail & Related papers (2024-03-25T00:15:34Z) - Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts.
Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor.
Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
arXiv Detail & Related papers (2024-02-18T07:49:22Z) - Energy-based Out-of-Distribution Detection for Graph Neural Networks [76.0242218180483]
We propose a simple, powerful and efficient OOD detection model for GNN-based learning on graphs, which we call GNNSafe.
GNNSafe achieves up to $17.0%$ AUROC improvement over state-of-the-arts and it could serve as simple yet strong baselines in such an under-developed area.
arXiv Detail & Related papers (2023-02-06T16:38:43Z) - Invariance Principle Meets Out-of-Distribution Generalization on Graphs [66.04137805277632]
Complex nature of graphs thwarts the adoption of the invariance principle for OOD generalization.
domain or environment partitions, which are often required by OOD methods, can be expensive to obtain for graphs.
We propose a novel framework to explicitly model this process using a contrastive strategy.
arXiv Detail & Related papers (2022-02-11T04:38:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.