Federated Graph Condensation with Information Bottleneck Principles

• URL: http://arxiv.org/abs/2405.03911v3
• Date: Mon, 18 Nov 2024 07:17:56 GMT
• Title: Federated Graph Condensation with Information Bottleneck Principles
• Authors: Bo Yan, Sihao He, Cheng Yang, Shang Liu, Yang Cao, Chuan Shi,
• Abstract summary: We propose and study the novel problem of federated graph condensation for graph neural networks (GNNs) We decouple the typical gradient matching process for graph condensation into client-side gradient calculation and server-side gradient matching. Our framework can consistently protect membership privacy during training.
• Score: 44.404509071881364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph condensation, which reduces the size of a large-scale graph by synthesizing a small-scale condensed graph as its substitution, has immediately benefited various graph learning tasks. However, existing graph condensation methods rely on centralized data storage, which is unfeasible for real-world decentralized data distribution, and overlook data holders' privacy-preserving requirements. To bridge the gap, we propose and study the novel problem of federated graph condensation for graph neural networks (GNNs). Specifically, we first propose a general framework for federated graph condensation, in which we decouple the typical gradient matching process for graph condensation into client-side gradient calculation and server-side gradient matching. In this way, the burdensome computation cost in client-side is largely alleviated. Besides, our empirical studies show that under the federated setting, the condensed graph will consistently leak data membership privacy, i.e., the condensed graph during the federated training can be utilized to steal the training data under the membership inference attacks (MIA). To tackle this issue, we innovatively incorporate information bottleneck principles into the federated graph condensation, which only needs to extract partial node features in one local pre-training step and utilize the features during federated training. Extensive experiments on real-world datasets demonstrate that our framework can consistently protect membership privacy during training. Meanwhile, it also achieves comparable and even superior performance against existing centralized graph condensation and federated graph learning methods.

Related papers

• Contrastive Graph Condensation: Advancing Data Versatility through Self-Supervised Learning [47.74244053386216]
  Graph condensation is a promising solution to synthesize a compact, substitute graph of the large-scale original graph. We introduce Contrastive Graph Condensation (CTGC), which adopts a self-supervised surrogate task to extract critical, causal information from the original graph. CTGC excels in handling various downstream tasks with a limited number of labels, consistently outperforming state-of-the-art GC methods.
  arXiv  Detail & Related papers  (2024-11-26T03:01:22Z)
• Federated Hypergraph Learning: Hyperedge Completion with Local Differential Privacy [6.295242666794106]
  FedHGL is designed to collaboratively train a comprehensive hypergraph neural network across multiple clients. Cross-client feature aggregation is performed and distributed at the central server to ensure that this information can be utilized by the clients.
  arXiv  Detail & Related papers  (2024-08-09T16:31:41Z)
• Self-Supervised Contrastive Graph Clustering Network via Structural Information Fusion [15.293684479404092]
  We propose a novel deep graph clustering method called CGCN. Our approach introduces contrastive signals and deep structural information into the pre-training process. Our method has been experimentally validated on multiple real-world graph datasets.
  arXiv  Detail & Related papers  (2024-08-08T09:49:26Z)
• GC-Bench: An Open and Unified Benchmark for Graph Condensation [54.70801435138878]
  We develop a comprehensive Graph Condensation Benchmark (GC-Bench) to analyze the performance of graph condensation. GC-Bench systematically investigates the characteristics of graph condensation in terms of the following dimensions: effectiveness, transferability, and complexity. We have developed an easy-to-use library for training and evaluating different GC methods to facilitate reproducible research.
  arXiv  Detail & Related papers  (2024-06-30T07:47:34Z)
• RobGC: Towards Robust Graph Condensation [61.259453496191696]
  Graph neural networks (GNNs) have attracted widespread attention for their impressive capability of graph representation learning. However, the increasing prevalence of large-scale graphs presents a significant challenge for GNN training due to their computational demands. We propose graph condensation (GC) to generate an informative compact graph that enables efficient training of GNNs while retaining performance.
  arXiv  Detail & Related papers  (2024-06-19T04:14:57Z)
• Simple Graph Condensation [30.85754566420301]
  Graph condensation involves tuning Graph Neural Networks (GNNs) on a small condensed graph for use on a large-scale original graph. We introduce the Simple Graph Condensation (SimGC) framework, which aligns the condensed graph with the original graph from the input layer to the prediction layer. SimGC achieves a significant speedup of up to 10 times compared to existing graph condensation methods.
  arXiv  Detail & Related papers  (2024-03-22T05:04:48Z)
• Graph Data Condensation via Self-expressive Graph Structure Reconstruction [7.4525875528900665]
  We introduce a novel framework named textbfGraph Data textbfCondensation via textbfSelf-expressive Graph Structure textbfReconstruction. Our method explicitly incorporates the original graph structure into the condensing process and captures the nuanced interdependencies between the condensed nodes.
  arXiv  Detail & Related papers  (2024-03-12T03:54:25Z)
• Deep Contrastive Graph Learning with Clustering-Oriented Guidance [61.103996105756394]
  Graph Convolutional Network (GCN) has exhibited remarkable potential in improving graph-based clustering. Models estimate an initial graph beforehand to apply GCN. Deep Contrastive Graph Learning (DCGL) model is proposed for general data clustering.
  arXiv  Detail & Related papers  (2024-02-25T07:03:37Z)
• Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching [26.303436980548174]
  Graph condensation aims to reduce the size of a large-scale graph dataset by synthesizing a compact counterpart. Existing methods often fall short of accurately replicating the original graph for certain datasets. In this paper, we make the first attempt toward textitlossless graph condensation by bridging the previously neglected supervision signals.
  arXiv  Detail & Related papers  (2024-02-07T16:32:02Z)
• Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching [50.30124426442228]
  Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have raised growing concerns. We propose a novel graph method named textbfCraftextbfTing textbfRationatextbf (textbfCTRL) which offers an optimized starting point closer to the original dataset's feature distribution.
  arXiv  Detail & Related papers  (2024-02-07T14:49:10Z)
• PUMA: Efficient Continual Graph Learning for Node Classification with Graph Condensation [49.00940417190911]
  Existing graph representation learning models encounter a catastrophic problem when learning with newly incoming graphs. In this paper, we propose a PUdo-label guided Memory bAnkrogation (PUMA) framework to enhance its efficiency and effectiveness.
  arXiv  Detail & Related papers  (2023-12-22T05:09:58Z)
• Structure-free Graph Condensation: From Large-scale Graphs to Condensed Graph-free Data [91.27527985415007]
  Existing graph condensation methods rely on the joint optimization of nodes and structures in the condensed graph. We advocate a new Structure-Free Graph Condensation paradigm, named SFGC, to distill a large-scale graph into a small-scale graph node set.
  arXiv  Detail & Related papers  (2023-06-05T07:53:52Z)
• Lumos: Heterogeneity-aware Federated Graph Learning over Decentralized Devices [19.27111697495379]
  Graph neural networks (GNNs) have been widely deployed in real-world networked applications and systems. We propose the first federated GNN framework called Lumos that supports supervised and unsupervised learning. Based on the constructed tree for each client, a decentralized tree-based GNN trainer is proposed to support versatile training.
  arXiv  Detail & Related papers  (2023-03-01T13:27:06Z)
• FedEgo: Privacy-preserving Personalized Federated Graph Learning with Ego-graphs [22.649780281947837]
  In some practical scenarios, graph data are stored separately in multiple distributed parties, which may not be directly shared due to conflicts of interest. We propose FedEgo, a federated graph learning framework based on ego-graphs to tackle the challenges above.
  arXiv  Detail & Related papers  (2022-08-29T15:47:36Z)
• GraphCoCo: Graph Complementary Contrastive Learning [65.89743197355722]
  Graph Contrastive Learning (GCL) has shown promising performance in graph representation learning (GRL) without the supervision of manual annotations. This paper proposes an effective graph complementary contrastive learning approach named GraphCoCo to tackle the above issue.
  arXiv  Detail & Related papers  (2022-03-24T02:58:36Z)
• Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations [94.41860307845812]
  Self-supervision is recently surging at its new frontier of graph learning. GraphCL uses a prefabricated prior reflected by the ad-hoc manual selection of graph data augmentations. We have extended the prefabricated discrete prior in the augmentation set, to a learnable continuous prior in the parameter space of graph generators. We have leveraged both principles of information minimization (InfoMin) and information bottleneck (InfoBN) to regularize the learned priors.
  arXiv  Detail & Related papers  (2022-01-04T15:49:18Z)
• Distributed Graph Learning with Smooth Data Priors [61.405131495287755]
  We propose a novel distributed graph learning algorithm, which permits to infer a graph from signal observations on the nodes. Our results show that the distributed approach has a lower communication cost than a centralised algorithm without compromising the accuracy in the inferred graph.
  arXiv  Detail & Related papers  (2021-12-11T00:52:02Z)
• Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning [21.0019144298605]
  Existing graph neural networks fed with the complete graph data are not scalable due to limited computation and memory costs. textscSubg-Con is proposed by utilizing the strong correlation between central nodes and their sampled subgraphs to capture regional structure information. Compared with existing graph representation learning approaches, textscSubg-Con has prominent performance advantages in weaker supervision requirements, model learning scalability, and parallelization.
  arXiv  Detail & Related papers  (2020-09-22T01:58:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.