Causal-aware Graph Neural Architecture Search under Distribution Shifts

• URL: http://arxiv.org/abs/2405.16489v2
• Date: Mon, 30 Dec 2024 11:28:31 GMT
• Title: Causal-aware Graph Neural Architecture Search under Distribution Shifts
• Authors: Peiwen Li, Xin Wang, Zeyang Zhang, Yijian Qin, Ziwei Zhang, Jialong Wang, Yang Li, Wenwu Zhu,
• Abstract summary: Causal-aware Graph Neural Architecture Search (CARNAS) is able to capture the causal graph-architecture relationship during the architecture search process. We propose Graph Embedding Intervention to intervene on causal subgraphs within the latent space.
• Score: 48.02254981004058
• License:
• Abstract: Graph NAS has emerged as a promising approach for autonomously designing GNN architectures by leveraging the correlations between graphs and architectures. Existing methods fail to generalize under distribution shifts that are ubiquitous in real-world graph scenarios, mainly because the graph-architecture correlations they exploit might be spurious and varying across distributions. We propose to handle the distribution shifts in the graph architecture search process by discovering and exploiting the causal relationship between graphs and architectures to search for the optimal architectures that can generalize under distribution shifts. The problem remains unexplored with following challenges: how to discover the causal graph-architecture relationship that has stable predictive abilities across distributions, and how to handle distribution shifts with the discovered causal graph-architecture relationship to search the generalized graph architectures. To address these challenges, we propose Causal-aware Graph Neural Architecture Search (CARNAS), which is able to capture the causal graph-architecture relationship during the architecture search process and discover the generalized graph architecture under distribution shifts. Specifically, we propose Disentangled Causal Subgraph Identification to capture the causal subgraphs that have stable prediction abilities across distributions. Then, we propose Graph Embedding Intervention to intervene on causal subgraphs within the latent space, ensuring that these subgraphs encapsulate essential features for prediction while excluding non-causal elements. Additionally, we propose Invariant Architecture Customization to reinforce the causal invariant nature of the causal subgraphs, which are utilized to tailor generalized graph architectures. Extensive experiments demonstrate that CARNAS achieves advanced out-of-distribution generalization ability.

Related papers

• Generative Risk Minimization for Out-of-Distribution Generalization on Graphs [71.48583448654522]
  We propose an innovative framework, named Generative Risk Minimization (GRM), designed to generate an invariant subgraph for each input graph to be classified, instead of extraction. We conduct extensive experiments across a variety of real-world graph datasets for both node-level and graph-level OOD generalization.
  arXiv  Detail & Related papers  (2025-02-11T21:24:13Z)
• Do Graph Diffusion Models Accurately Capture and Generate Substructure Distributions? [28.19526635775658]
  Diffusion models do not possess universal expressivity to accurately model the distribution scores of complex graph data. Our work addresses this limitation by focusing on the frequency of specific substructures as a key characteristic of target graph distributions. We establish a theoretical connection between the expressivity of Graph Neural Networks (GNNs) and the overall performance of graph diffusion models.
  arXiv  Detail & Related papers  (2025-02-04T17:04:16Z)
• Graph Size-imbalanced Learning with Energy-guided Structural Smoothing [13.636616140250908]
  Real-world graphs usually suffer from the size-imbalanced problem in the multi-graph classification. Recent studies find that off-the-shelf Graph Neural Networks (GNNs) would compromise model performance under the long-tailed settings. We propose a novel energy-based size-imbalanced learning framework named textbfSIMBA, which smooths the features between head and tail graphs.
  arXiv  Detail & Related papers  (2024-12-23T14:06:49Z)
• GraphCroc: Cross-Correlation Autoencoder for Graph Structural Reconstruction [6.817416560637197]
  Graph autoencoders (GAEs) reconstruct graph structures from node embeddings. We introduce a cross-correlation mechanism that significantly enhances the GAE representational capabilities. We also propose GraphCroc, a new GAE that supports flexible encoder architectures tailored for various downstream tasks.
  arXiv  Detail & Related papers  (2024-10-04T12:59:45Z)
• AnyGraph: Graph Foundation Model in the Wild [16.313146933922752]
  Graph foundation models offer the potential to learn robust, generalizable representations from graph data. In this work, we investigate a unified graph model, AnyGraph, designed to handle key challenges. Our experiments on diverse 38 graph datasets have demonstrated the strong zero-shot learning performance of AnyGraph.
  arXiv  Detail & Related papers  (2024-08-20T09:57:13Z)
• Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision [51.88848982611515]
  Unsupervised graph neural architecture search remains unexplored in the literature. We propose a novel Disentangled Self-supervised Graph Neural Architecture Search model. Our model is able to achieve state-of-the-art performance against several baseline methods in an unsupervised manner.
  arXiv  Detail & Related papers  (2024-03-08T05:23:55Z)
• GrannGAN: Graph annotation generative adversarial networks [72.66289932625742]
  We consider the problem of modelling high-dimensional distributions and generating new examples of data with complex relational feature structure coherent with a graph skeleton. The model we propose tackles the problem of generating the data features constrained by the specific graph structure of each data point by splitting the task into two phases. In the first it models the distribution of features associated with the nodes of the given graph, in the second it complements the edge features conditionally on the node features.
  arXiv  Detail & Related papers  (2022-12-01T11:49:07Z)
• Graph Condensation via Receptive Field Distribution Matching [61.71711656856704]
  This paper focuses on creating a small graph to represent the original graph, so that GNNs trained on the size-reduced graph can make accurate predictions. We view the original graph as a distribution of receptive fields and aim to synthesize a small graph whose receptive fields share a similar distribution.
  arXiv  Detail & Related papers  (2022-06-28T02:10:05Z)
• GraphOpt: Learning Optimization Models of Graph Formation [72.75384705298303]
  We propose an end-to-end framework that learns an implicit model of graph structure formation and discovers an underlying optimization mechanism. The learned objective can serve as an explanation for the observed graph properties, thereby lending itself to transfer across different graphs within a domain. GraphOpt poses link formation in graphs as a sequential decision-making process and solves it using maximum entropy inverse reinforcement learning algorithm.
  arXiv  Detail & Related papers  (2020-07-07T16:51:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.