Invariance Principle Meets Out-of-Distribution Generalization on Graphs

• URL: http://arxiv.org/abs/2202.05441v1
• Date: Fri, 11 Feb 2022 04:38:39 GMT
• Title: Invariance Principle Meets Out-of-Distribution Generalization on Graphs
• Authors: Yongqiang Chen, Yonggang Zhang, Han Yang, Kaili Ma, Binghui Xie, Tongliang Liu, Bo Han, James Cheng
• Abstract summary: Complex nature of graphs thwarts the adoption of the invariance principle for OOD generalization. domain or environment partitions, which are often required by OOD methods, can be expensive to obtain for graphs. We propose a novel framework to explicitly model this process using a contrastive strategy.
• Score: 66.04137805277632
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite recent developments in using the invariance principle from causality to enable out-of-distribution (OOD) generalization on Euclidean data, e.g., images, studies on graph data are limited. Different from images, the complex nature of graphs poses unique challenges that thwart the adoption of the invariance principle for OOD generalization. In particular, distribution shifts on graphs can happen at both structure-level and attribute-level, which increases the difficulty of capturing the invariance. Moreover, domain or environment partitions, which are often required by OOD methods developed on Euclidean data, can be expensive to obtain for graphs. Aiming to bridge this gap, we characterize distribution shifts on graphs with causal models, and show that the OOD generalization on graphs with invariance principle is possible by identifying an invariant subgraph for making predictions. We propose a novel framework to explicitly model this process using a contrastive strategy. By contrasting the estimated invariant subgraphs, our framework can provably identify the underlying invariant subgraph under mild assumptions. Experiments across several synthetic and real-world datasets demonstrate the state-of-the-art OOD generalization ability of our method.

Related papers

• Subgraph Aggregation for Out-of-Distribution Generalization on Graphs [29.884717215947745]
  Out-of-distribution (OOD) generalization in Graph Neural Networks (GNNs) has gained significant attention. We propose a novel framework, SubGraph Aggregation (SuGAr), designed to learn a diverse set of subgraphs. Experiments on both synthetic and real-world datasets demonstrate that SuGAr outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-29T16:54:37Z)
• IENE: Identifying and Extrapolating the Node Environment for Out-of-Distribution Generalization on Graphs [10.087216264788097]
  We propose IENE, an OOD generalization method on graphs based on node-level environmental identification and extrapolation techniques. It strengthens the model's ability to extract invariance from two granularities simultaneously, leading to improved generalization.
  arXiv  Detail & Related papers  (2024-06-02T14:43:56Z)
• Graphs Generalization under Distribution Shifts [11.963958151023732]
  We introduce a novel framework, namely Graph Learning Invariant Domain genERation (GLIDER) Our model outperforms baseline methods on node-level OOD generalization across domains in distribution shift on node features and topological structures simultaneously.
  arXiv  Detail & Related papers  (2024-03-25T00:15:34Z)
• Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
  We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts. Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor. Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
  arXiv  Detail & Related papers  (2024-02-18T07:49:22Z)
• Unifying Invariance and Spuriousity for Graph Out-of-Distribution via Probability of Necessity and Sufficiency [19.49531172542614]
  We propose a unified framework to exploit the Probability of Necessity and Sufficiency to extract the Invariant Substructure (PNSIS) Our model outperforms the state-of-the-art techniques on graph OOD on several benchmarks.
  arXiv  Detail & Related papers  (2024-02-14T13:31:53Z)
• Graph Out-of-Distribution Generalization with Controllable Data Augmentation [51.17476258673232]
  Graph Neural Network (GNN) has demonstrated extraordinary performance in classifying graph properties. Due to the selection bias of training and testing data, distribution deviation is widespread. We propose OOD calibration to measure the distribution deviation of virtual samples.
  arXiv  Detail & Related papers  (2023-08-16T13:10:27Z)
• Graph Structure and Feature Extrapolation for Out-of-Distribution Generalization [54.64375566326931]
  Out-of-distribution (OOD) generalization deals with the prevalent learning scenario where test distribution shifts from training distribution. We propose to achieve graph OOD generalization with the novel design of non-Euclidean-space linear extrapolation. Our design tailors OOD samples for specific shifts without corrupting underlying causal mechanisms.
  arXiv  Detail & Related papers  (2023-06-13T18:46:28Z)
• Handling Distribution Shifts on Graphs: An Invariance Perspective [78.31180235269035]
  We formulate the OOD problem on graphs and develop a new invariant learning approach, Explore-to-Extrapolate Risk Minimization (EERM) EERM resorts to multiple context explorers that are adversarially trained to maximize the variance of risks from multiple virtual environments. We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
  arXiv  Detail & Related papers  (2022-02-05T02:31:01Z)
• A Robust and Generalized Framework for Adversarial Graph Embedding [73.37228022428663]
  We propose a robust framework for adversarial graph embedding, named AGE. AGE generates the fake neighbor nodes as the enhanced negative samples from the implicit distribution. Based on this framework, we propose three models to handle three types of graph data.
  arXiv  Detail & Related papers  (2021-05-22T07:05:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.