Federated Graph Condensation with Information Bottleneck Principles

• URL: http://arxiv.org/abs/2405.03911v4
• Date: Fri, 20 Dec 2024 07:57:07 GMT
• Title: Federated Graph Condensation with Information Bottleneck Principles
• Authors: Bo Yan, Sihao He, Cheng Yang, Shang Liu, Yang Cao, Chuan Shi,
• Abstract summary: We propose and study the novel problem of federated graph condensation (FGC) for graph neural networks (GNNs) Under the federated setting, the condensed graph will consistently leak data membership privacy. Our framework consistently protects membership privacy during training.
• Score: 44.404509071881364
• License:
• Abstract: Graph condensation (GC), which reduces the size of a large-scale graph by synthesizing a small-scale condensed graph as its substitution, has benefited various graph learning tasks. However, existing GC methods rely on centralized data storage, which is unfeasible for real-world decentralized data distribution, and overlook data holders' privacy-preserving requirements. To bridge this gap, we propose and study the novel problem of federated graph condensation (FGC) for graph neural networks (GNNs). Specifically, we first propose a general framework for FGC, where we decouple the typical gradient matching process for GC into client-side gradient calculation and server-side gradient matching, integrating knowledge from multiple clients' subgraphs into one smaller condensed graph. Nevertheless, our empirical studies show that under the federated setting, the condensed graph will consistently leak data membership privacy, i.e., the condensed graph during federated training can be utilized to steal training data under the membership inference attack (MIA). To tackle this issue, we innovatively incorporate information bottleneck principles into the FGC, which only needs to extract partial node features in one local pre-training step and utilize the features during federated training. Theoretical and experimental analyses demonstrate that our framework consistently protects membership privacy during training. Meanwhile, it can achieve comparable and even superior performance against existing centralized GC and federated graph learning (FGL) methods.

Related papers

• Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning [47.74244053386216]
  Graph condensation is a promising solution to synthesize a compact, substitute graph of the large-scale original graph. We introduce Contrastive Graph Condensation (CTGC), which adopts a self-supervised surrogate task to extract critical, causal information from the original graph. CTGC excels in handling various downstream tasks with a limited number of labels, consistently outperforming state-of-the-art GC methods.
  arXiv  Detail & Related papers  (2024-11-26T03:01:22Z)
• Federated Hypergraph Learning: Hyperedge Completion with Local Differential Privacy [6.295242666794106]
  FedHGL is designed to collaboratively train a comprehensive hypergraph neural network across multiple clients. Cross-client feature aggregation is performed and distributed at the central server to ensure that this information can be utilized by the clients.
  arXiv  Detail & Related papers  (2024-08-09T16:31:41Z)
• Self-Supervised Contrastive Graph Clustering Network via Structural Information Fusion [15.293684479404092]
  We propose a novel deep graph clustering method called CGCN. Our approach introduces contrastive signals and deep structural information into the pre-training process. Our method has been experimentally validated on multiple real-world graph datasets.
  arXiv  Detail & Related papers  (2024-08-08T09:49:26Z)
• GC-Bench: An Open and Unified Benchmark for Graph Condensation [54.70801435138878]
  We develop a comprehensive Graph Condensation Benchmark (GC-Bench) to analyze the performance of graph condensation. GC-Bench systematically investigates the characteristics of graph condensation in terms of the following dimensions: effectiveness, transferability, and complexity. We have developed an easy-to-use library for training and evaluating different GC methods to facilitate reproducible research.
  arXiv  Detail & Related papers  (2024-06-30T07:47:34Z)
• RobGC: Towards Robust Graph Condensation [61.259453496191696]
  Graph neural networks (GNNs) have attracted widespread attention for their impressive capability of graph representation learning. However, the increasing prevalence of large-scale graphs presents a significant challenge for GNN training due to their computational demands. We propose graph condensation (GC) to generate an informative compact graph that enables efficient training of GNNs while retaining performance.
  arXiv  Detail & Related papers  (2024-06-19T04:14:57Z)
• Deep Contrastive Graph Learning with Clustering-Oriented Guidance [61.103996105756394]
  Graph Convolutional Network (GCN) has exhibited remarkable potential in improving graph-based clustering. Models estimate an initial graph beforehand to apply GCN. Deep Contrastive Graph Learning (DCGL) model is proposed for general data clustering.
  arXiv  Detail & Related papers  (2024-02-25T07:03:37Z)
• Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data [91.27527985415007]
  Existing graph condensation methods rely on the joint optimization of nodes and structures in the condensed graph. We advocate a new Structure-Free Graph Condensation paradigm, named SFGC, to distill a large-scale graph into a small-scale graph node set.
  arXiv  Detail & Related papers  (2023-06-05T07:53:52Z)
• Lumos: Heterogeneity-aware Federated Graph Learning over Decentralized Devices [19.27111697495379]
  Graph neural networks (GNNs) have been widely deployed in real-world networked applications and systems. We propose the first federated GNN framework called Lumos that supports supervised and unsupervised learning. Based on the constructed tree for each client, a decentralized tree-based GNN trainer is proposed to support versatile training.
  arXiv  Detail & Related papers  (2023-03-01T13:27:06Z)
• FedEgo: Privacy-preserving Personalized Federated Graph Learning with Ego-graphs [22.649780281947837]
  In some practical scenarios, graph data are stored separately in multiple distributed parties, which may not be directly shared due to conflicts of interest. We propose FedEgo, a federated graph learning framework based on ego-graphs to tackle the challenges above.
  arXiv  Detail & Related papers  (2022-08-29T15:47:36Z)
• GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
  Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
  arXiv  Detail & Related papers  (2022-03-24T02:58:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.