Unifying Invariant and Variant Features for Graph Out-of-Distribution via Probability of Necessity and Sufficiency

• URL: http://arxiv.org/abs/2407.15273v1
• Date: Sun, 21 Jul 2024 21:35:01 GMT
• Title: Unifying Invariant and Variant Features for Graph Out-of-Distribution via Probability of Necessity and Sufficiency
• Authors: Xuexin Chen, Ruichu Cai, Kaitao Zheng, Zhifan Jiang, Zhengting Huang, Zhifeng Hao, Zijian Li,
• Abstract summary: We propose exploiting Probability of Necessity and Sufficiency (PNS) to extract sufficient and necessary invariant substructures. We also leverage the domain variant subgraphs related to the labels to boost the generalization performance in an ensemble manner. Experimental results demonstrate that our SNIGL model outperforms the state-of-the-art techniques on six public benchmarks.
• Score: 18.564387153282293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph Out-of-Distribution (OOD), requiring that models trained on biased data generalize to the unseen test data, has considerable real-world applications. One of the most mainstream methods is to extract the invariant subgraph by aligning the original and augmented data with the help of environment augmentation. However, these solutions might lead to the loss or redundancy of semantic subgraphs and result in suboptimal generalization. To address this challenge, we propose exploiting Probability of Necessity and Sufficiency (PNS) to extract sufficient and necessary invariant substructures. Beyond that, we further leverage the domain variant subgraphs related to the labels to boost the generalization performance in an ensemble manner. Specifically, we first consider the data generation process for graph data. Under mild conditions, we show that the sufficient and necessary invariant subgraph can be extracted by minimizing an upper bound, built on the theoretical advance of the probability of necessity and sufficiency. To further bridge the theory and algorithm, we devise the model called Sufficiency and Necessity Inspired Graph Learning (SNIGL), which ensembles an invariant subgraph classifier on top of latent sufficient and necessary invariant subgraphs, and a domain variant subgraph classifier specific to the test domain for generalization enhancement. Experimental results demonstrate that our SNIGL model outperforms the state-of-the-art techniques on six public benchmarks, highlighting its effectiveness in real-world scenarios.

Related papers

• HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters [53.97380482341493]
  "pre-train, prompt-tuning" has demonstrated impressive performance for tuning pre-trained heterogeneous graph neural networks (HGNNs) We propose a unified framework that combines two new adapters with potential labeled data extension to improve the generalization of pre-trained HGNN models.
  arXiv  Detail & Related papers  (2024-11-02T06:43:54Z)
• Subgraph Aggregation for Out-of-Distribution Generalization on Graphs [29.884717215947745]
  Out-of-distribution (OOD) generalization in Graph Neural Networks (GNNs) has gained significant attention. We propose a novel framework, SubGraph Aggregation (SuGAr), designed to learn a diverse set of subgraphs. Experiments on both synthetic and real-world datasets demonstrate that SuGAr outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-29T16:54:37Z)
• Unifying Invariance and Spuriousity for Graph Out-of-Distribution via Probability of Necessity and Sufficiency [19.49531172542614]
  We propose a unified framework to exploit the Probability of Necessity and Sufficiency to extract the Invariant Substructure (PNSIS) Our model outperforms the state-of-the-art techniques on graph OOD on several benchmarks.
  arXiv  Detail & Related papers  (2024-02-14T13:31:53Z)
• Invariant Graph Transformer [0.0]
  In graph machine learning context, graph rationalization can enhance the model performance. A key technique named "intervention" is applied to ensure the discriminative power of the extracted rationale subgraphs. In this paper, we propose well-tailored intervention strategies on graph data.
  arXiv  Detail & Related papers  (2023-12-13T02:56:26Z)
• GraphGLOW: Universal and Generalizable Structure Learning for Graph Neural Networks [72.01829954658889]
  This paper introduces the mathematical definition of this novel problem setting. We devise a general framework that coordinates a single graph-shared structure learner and multiple graph-specific GNNs. The well-trained structure learner can directly produce adaptive structures for unseen target graphs without any fine-tuning.
  arXiv  Detail & Related papers  (2023-06-20T03:33:22Z)
• Joint Graph Learning and Model Fitting in Laplacian Regularized Stratified Models [5.933030735757292]
  Laplacian regularized stratified models (LRSM) are models that utilize the explicit or implicit network structure of the sub-problems. This paper shows the importance and sensitivity of graph weights in LRSM, and provably show that the sensitivity can be arbitrarily large. We propose a generic approach to jointly learn the graph while fitting the model parameters by solving a single optimization problem.
  arXiv  Detail & Related papers  (2023-05-04T06:06:29Z)
• Optimal Propagation for Graph Neural Networks [51.08426265813481]
  We propose a bi-level optimization approach for learning the optimal graph structure. We also explore a low-rank approximation model for further reducing the time complexity.
  arXiv  Detail & Related papers  (2022-05-06T03:37:00Z)
• Invariance Principle Meets Out-of-Distribution Generalization on Graphs [66.04137805277632]
  Complex nature of graphs thwarts the adoption of the invariance principle for OOD generalization. domain or environment partitions, which are often required by OOD methods, can be expensive to obtain for graphs. We propose a novel framework to explicitly model this process using a contrastive strategy.
  arXiv  Detail & Related papers  (2022-02-11T04:38:39Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• Crime Prediction with Graph Neural Networks and Multivariate Normal Distributions [18.640610803366876]
  We tackle the sparsity problem in high resolution by leveraging the flexible structure of graph convolutional networks (GCNs) We build our model with Graph Convolutional Gated Recurrent Units (Graph-ConvGRU) to learn spatial, temporal, and categorical relations. We show that our model is not only generative but also precise.
  arXiv  Detail & Related papers  (2021-11-29T17:37:01Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.