MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning

• URL: http://arxiv.org/abs/2307.13055v3
• Date: Wed, 2 Aug 2023 14:30:57 GMT
• Title: MARIO: Model Agnostic Recipe for Improving OOD Generalization of Graph Contrastive Learning
• Authors: Yun Zhu, Haizhou Shi, Zhenshuo Zhang, Siliang Tang
• Abstract summary: We investigate the problem of out-of-distribution (OOD) generalization for unsupervised learning methods on graph data. We propose a underlineModel-underlineAgnostic underlineRecipe for underlineImproving underlineOOD generalizability.
• Score: 18.744939223003673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this work, we investigate the problem of out-of-distribution (OOD) generalization for unsupervised learning methods on graph data. This scenario is particularly challenging because graph neural networks (GNNs) have been shown to be sensitive to distributional shifts, even when labels are available. To address this challenge, we propose a \underline{M}odel-\underline{A}gnostic \underline{R}ecipe for \underline{I}mproving \underline{O}OD generalizability of unsupervised graph contrastive learning methods, which we refer to as MARIO. MARIO introduces two principles aimed at developing distributional-shift-robust graph contrastive methods to overcome the limitations of existing frameworks: (i) Information Bottleneck (IB) principle for achieving generalizable representations and (ii) Invariant principle that incorporates adversarial data augmentation to obtain invariant representations. To the best of our knowledge, this is the first work that investigates the OOD generalization problem of graph contrastive learning, with a specific focus on node-level tasks. Through extensive experiments, we demonstrate that our method achieves state-of-the-art performance on the OOD test set, while maintaining comparable performance on the in-distribution test set when compared to existing approaches. The source code for our method can be found at: https://github.com/ZhuYun97/MARIO

Related papers

• DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalization [44.291382840373]
  This paper addresses the challenge of out-of-distribution generalization in graph machine learning. Traditional graph learning algorithms falter in real-world scenarios where this assumption fails. A principal factor contributing to this suboptimal performance is the inherent simplicity bias of neural networks.
  arXiv  Detail & Related papers  (2024-08-08T12:08:55Z)
• Topology-Aware Dynamic Reweighting for Distribution Shifts on Graph [24.44321658238713]
  Graph Neural Networks (GNNs) are widely used for node classification tasks but often fail to generalize when training and test nodes come from different distributions. We introduce the Topology-Aware Dynamic Reweighting (TAR) framework, which dynamically adjusts sample weights through gradient flow in the Wasserstein space during training. Our framework's superiority is demonstrated through standard testing on four graph OOD datasets and three class-imbalanced node classification datasets.
  arXiv  Detail & Related papers  (2024-06-03T07:32:05Z)
• Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
  We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts. Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor. Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
  arXiv  Detail & Related papers  (2024-02-18T07:49:22Z)
• Rethinking Graph Masked Autoencoders through Alignment and Uniformity [26.86368034133612]
  Self-supervised learning on graphs can be bifurcated into contrastive and generative methods. Recent advent of graph masked autoencoder (GraphMAE) rekindles momentum behind generative methods.
  arXiv  Detail & Related papers  (2024-02-11T15:21:08Z)
• Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization [60.4169201192582]
  We propose to incorporate label and environment causal independence (LECI) to fully make use of label and environment information. LECI significantly outperforms prior methods on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-06-01T19:33:30Z)
• ARIEL: Adversarial Graph Contrastive Learning [51.14695794459399]
  ARIEL consistently outperforms the current graph contrastive learning methods for both node-level and graph-level classification tasks. ARIEL is more robust in the face of adversarial attacks.
  arXiv  Detail & Related papers  (2022-08-15T01:24:42Z)
• Invariance Principle Meets Out-of-Distribution Generalization on Graphs [66.04137805277632]
  Complex nature of graphs thwarts the adoption of the invariance principle for OOD generalization. domain or environment partitions, which are often required by OOD methods, can be expensive to obtain for graphs. We propose a novel framework to explicitly model this process using a contrastive strategy.
  arXiv  Detail & Related papers  (2022-02-11T04:38:39Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• A Robust and Generalized Framework for Adversarial Graph Embedding [73.37228022428663]
  We propose a robust framework for adversarial graph embedding, named AGE. AGE generates the fake neighbor nodes as the enhanced negative samples from the implicit distribution. Based on this framework, we propose three models to handle three types of graph data.
  arXiv  Detail & Related papers  (2021-05-22T07:05:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.