Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphs

• URL: http://arxiv.org/abs/2401.03597v3
• Date: Tue, 16 Apr 2024 04:36:18 GMT
• Title: Few-Shot Causal Representation Learning for Out-of-Distribution Generalization on Heterogeneous Graphs
• Authors: Pengfei Ding, Yan Wang, Guanfeng Liu, Nan Wang, Xiaofang Zhou,
• Abstract summary: Heterogeneous graph few-shot learning (HGFL) has been developed to address the label sparsity issue in heterogeneous graphs (HGs) We propose a novel Causal OOD Heterogeneous graph Few-shot learning model, namely COHF.
• Score: 16.130356170284127
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Heterogeneous graph few-shot learning (HGFL) has been developed to address the label sparsity issue in heterogeneous graphs (HGs), which consist of various types of nodes and edges. The core concept of HGFL is to extract knowledge from rich-labeled classes in a source HG, transfer this knowledge to a target HG to facilitate learning new classes with few-labeled training data, and finally make predictions on unlabeled testing data. Existing methods typically assume that the source HG, training data, and testing data all share the same distribution. However, in practice, distribution shifts among these three types of data are inevitable due to two reasons: (1) the limited availability of the source HG that matches the target HG distribution, and (2) the unpredictable data generation mechanism of the target HG. Such distribution shifts result in ineffective knowledge transfer and poor learning performance in existing methods, thereby leading to a novel problem of out-of-distribution (OOD) generalization in HGFL. To address this challenging problem, we propose a novel Causal OOD Heterogeneous graph Few-shot learning model, namely COHF. In COHF, we first characterize distribution shifts in HGs with a structural causal model, establishing an invariance principle for OOD generalization in HGFL. Then, following this invariance principle, we propose a new variational autoencoder-based heterogeneous graph neural network to mitigate the impact of distribution shifts. Finally, by integrating this network with a novel meta-learning framework, COHF effectively transfers knowledge to the target HG to predict new classes with few-labeled data. Extensive experiments on seven real-world datasets have demonstrated the superior performance of COHF over the state-of-the-art methods.

Related papers

• FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning [12.834423184614849]
  Subgraph federated learning (subgraph-FL) facilitates the collaborative training of graph neural networks (GNNs) by multi-client subgraphs. node and topology variation leads to significant differences in the class-wise knowledge reliability of multiple local GNNs. We propose topology-aware data-free knowledge distillation technology (FedTAD) to enhance reliable knowledge transfer from the local model to the global model.
  arXiv  Detail & Related papers  (2024-04-22T10:19:02Z)
• Graph Out-of-Distribution Generalization via Causal Intervention [69.70137479660113]
  We introduce a conceptually simple yet principled approach for training robust graph neural networks (GNNs) under node-level distribution shifts. Our method resorts to a new learning objective derived from causal inference that coordinates an environment estimator and a mixture-of-expert GNN predictor. Our model can effectively enhance generalization with various types of distribution shifts and yield up to 27.4% accuracy improvement over state-of-the-arts on graph OOD generalization benchmarks.
  arXiv  Detail & Related papers  (2024-02-18T07:49:22Z)
• HGAttack: Transferable Heterogeneous Graph Adversarial Attack [63.35560741500611]
  Heterogeneous Graph Neural Networks (HGNNs) are increasingly recognized for their performance in areas like the web and e-commerce. This paper introduces HGAttack, the first dedicated gray box evasion attack method for heterogeneous graphs.
  arXiv  Detail & Related papers  (2024-01-18T12:47:13Z)
• HGPROMPT: Bridging Homogeneous and Heterogeneous Graphs for Few-shot Prompt Learning [16.587427365950838]
  We propose HGPROMPT, a novel pre-training and prompting framework to unify not only pre-training and downstream tasks but also homogeneous and heterogeneous graphs. We thoroughly evaluate and analyze HGPROMPT through extensive experiments on three public datasets.
  arXiv  Detail & Related papers  (2023-12-04T13:20:15Z)
• Cross-heterogeneity Graph Few-shot Learning [9.80898395055038]
  We propose a novel model for Cross-heterogeneity Graph Few-shot Learning, namely CGFL. In CGFL, we first extract meta-patterns to capture heterogeneous information. Then, we propose a score module to measure the informativeness of labeled samples and determine the transferability of each source HG.
  arXiv  Detail & Related papers  (2023-08-10T01:25:28Z)
• Heterogeneous Graph Neural Networks using Self-supervised Reciprocally Contrastive Learning [102.9138736545956]
  Heterogeneous graph neural network (HGNN) is a very popular technique for the modeling and analysis of heterogeneous graphs. We develop for the first time a novel and robust heterogeneous graph contrastive learning approach, namely HGCL, which introduces two views on respective guidance of node attributes and graph topologies. In this new approach, we adopt distinct but most suitable attribute and topology fusion mechanisms in the two views, which are conducive to mining relevant information in attributes and topologies separately.
  arXiv  Detail & Related papers  (2022-04-30T12:57:02Z)
• Handling Distribution Shifts on Graphs: An Invariance Perspective [78.31180235269035]
  We formulate the OOD problem on graphs and develop a new invariant learning approach, Explore-to-Extrapolate Risk Minimization (EERM) EERM resorts to multiple context explorers that are adversarially trained to maximize the variance of risks from multiple virtual environments. We prove the validity of our method by theoretically showing its guarantee of a valid OOD solution.
  arXiv  Detail & Related papers  (2022-02-05T02:31:01Z)
• Generalizing Graph Neural Networks on Out-Of-Distribution Graphs [51.33152272781324]
  Graph Neural Networks (GNNs) are proposed without considering the distribution shifts between training and testing graphs. In such a setting, GNNs tend to exploit subtle statistical correlations existing in the training set for predictions, even though it is a spurious correlation. We propose a general causal representation framework, called StableGNN, to eliminate the impact of spurious correlations.
  arXiv  Detail & Related papers  (2021-11-20T18:57:18Z)
• Issues with Propagation Based Models for Graph-Level Outlier Detection [16.980621769406916]
  Graph-Level Outlier Detection ( GLOD) is the task of identifying unusual graphs within a graph database. This paper identifies and delves into a fundamental and intriguing issue with applying propagation based models to GLOD. We find that ROC-AUC performance of the models change significantly depending on which class is down-sampled.
  arXiv  Detail & Related papers  (2020-12-23T19:38:21Z)
• A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources [79.48829365560788]
  Heterogeneous graphs (HGs) also known as heterogeneous information networks have become ubiquitous in real-world scenarios. HG embedding aims to learn representations in a lower-dimension space while preserving the heterogeneous structures and semantics for downstream tasks.
  arXiv  Detail & Related papers  (2020-11-30T15:03:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.