PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN Training Method

• URL: http://arxiv.org/abs/2110.12906v1
• Date: Fri, 22 Oct 2021 08:22:36 GMT
• Title: PPSGCN: A Privacy-Preserving Subgraph Sampling Based Distributed GCN Training Method
• Authors: Binchi Zhang, Minnan Luo, Shangbin Feng, Ziqi Liu, Jun Zhou, Qinghua Zheng
• Abstract summary: Graph convolutional networks (GCNs) have been widely adopted for graph representation learning and achieved impressive performance. Existing methods directly exchange node features between different clients, which results in data privacy leakage. We propose a Privacy-Preserving Subgraph sampling based distributed GCN training method (PPSGCN) which preserves data privacy and significantly cuts back on communication and memory overhead.
• Score: 28.829761038950707
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Graph convolutional networks (GCNs) have been widely adopted for graph representation learning and achieved impressive performance. For larger graphs stored separately on different clients, distributed GCN training algorithms were proposed to improve efficiency and scalability. However, existing methods directly exchange node features between different clients, which results in data privacy leakage. Federated learning was incorporated in graph learning to tackle data privacy, while they suffer from severe performance drop due to non-iid data distribution. Besides, these approaches generally involve heavy communication and memory overhead during the training process. In light of these problems, we propose a Privacy-Preserving Subgraph sampling based distributed GCN training method (PPSGCN), which preserves data privacy and significantly cuts back on communication and memory overhead. Specifically, PPSGCN employs a star-topology client-server system. We firstly sample a local node subset in each client to form a global subgraph, which greatly reduces communication and memory costs. We then conduct local computation on each client with features or gradients of the sampled nodes. Finally, all clients securely communicate with the central server with homomorphic encryption to combine local results while preserving data privacy. Compared with federated graph learning methods, our PPSGCN model is trained on a global graph to avoid the negative impact of local data distribution. We prove that our PPSGCN algorithm would converge to a local optimum with probability 1. Experiment results on three prevalent benchmarks demonstrate that our algorithm significantly reduces communication and memory overhead while maintaining desirable performance. Further studies not only demonstrate the fast convergence of PPSGCN, but discuss the trade-off between communication and local computation cost as well.

Related papers

• FedHERO: A Federated Learning Approach for Node Classification Task on Heterophilic Graphs [55.51300642911766]
  Federated Graph Learning (FGL) empowers clients to collaboratively train Graph neural networks (GNNs) in a distributed manner. FGL methods usually require that the graph data owned by all clients is homophilic to ensure similar neighbor distribution patterns of nodes. We propose FedHERO, an FGL framework designed to harness and share insights from heterophilic graphs effectively.
  arXiv  Detail & Related papers  (2025-04-29T22:23:35Z)
• Subgraph Federated Learning for Local Generalization [41.64806982207585]
  Federated Learning (FL) on graphs enables collaborative model training to enhance performance without compromising privacy of each client. Existing methods often overlook the mutable nature of graph data, which frequently introduces new nodes and leads to shifts in label distribution. Our proposed method, FedLoG, effectively tackles this issue by mitigating local overfitting.
  arXiv  Detail & Related papers  (2025-03-06T01:08:01Z)
• SpreadFGL: Edge-Client Collaborative Federated Graph Learning with Adaptive Neighbor Generation [16.599474223790843]
  Federated Graph Learning (FGL) has garnered widespread attention by enabling collaborative training on multiple clients for classification tasks. We propose a novel FGL framework, named SpreadFGL, to promote the information flow in edge-client collaboration. We show that SpreadFGL achieves higher accuracy and faster convergence against state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-07-14T09:34:19Z)
• FedDistill: Global Model Distillation for Local Model De-Biasing in Non-IID Federated Learning [10.641875933652647]
  Federated Learning (FL) is a novel approach that allows for collaborative machine learning. FL faces challenges due to non-uniformly distributed (non-iid) data across clients. This paper introduces FedDistill, a framework enhancing the knowledge transfer from the global model to local models.
  arXiv  Detail & Related papers  (2024-04-14T10:23:30Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Federated Skewed Label Learning with Logits Fusion [23.062650578266837]
  Federated learning (FL) aims to collaboratively train a shared model across multiple clients without transmitting their local data. We propose FedBalance, which corrects the optimization bias among local models by calibrating their logits. Our method can gain 13% higher average accuracy compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-11-14T14:37:33Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• SphereFed: Hyperspherical Federated Learning [22.81101040608304]
  Key challenge is the handling of non-i.i.d. data across multiple clients. We introduce the Hyperspherical Federated Learning (SphereFed) framework to address the non-i.i.d. issue. We show that the calibration solution can be computed efficiently and distributedly without direct access of local data.
  arXiv  Detail & Related papers  (2022-07-19T17:13:06Z)
• FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction [48.85303253333453]
  Federated learning (FL) allows multiple clients to collectively train a high-performance global model without sharing their private data. We propose a novel federated learning algorithm with local drift decoupling and correction (FedDC) Our FedDC only introduces lightweight modifications in the local training phase, in which each client utilizes an auxiliary local drift variable to track the gap between the local model parameter and the global model parameters. Experiment results and analysis demonstrate that FedDC yields expediting convergence and better performance on various image classification tasks.
  arXiv  Detail & Related papers  (2022-03-22T14:06:26Z)
• Acceleration of Federated Learning with Alleviated Forgetting in Local Training [61.231021417674235]
  Federated learning (FL) enables distributed optimization of machine learning models while protecting privacy. We propose FedReg, an algorithm to accelerate FL with alleviated knowledge forgetting in the local training stage. Our experiments demonstrate that FedReg not only significantly improves the convergence rate of FL, especially when the neural network architecture is deep.
  arXiv  Detail & Related papers  (2022-03-05T02:31:32Z)
• Data Selection for Efficient Model Update in Federated Learning [0.07614628596146598]
  We propose to reduce the amount of local data that is needed to train a global model. We do this by splitting the model into a lower part for generic feature extraction and an upper part that is more sensitive to the characteristics of the local data. Our experiments show that less than 1% of the local data can transfer the characteristics of the client data to the global model.
  arXiv  Detail & Related papers  (2021-11-05T14:07:06Z)
• A Bayesian Federated Learning Framework with Online Laplace Approximation [144.7345013348257]
  Federated learning allows multiple clients to collaboratively learn a globally shared model. We propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side. We achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.
  arXiv  Detail & Related papers  (2021-02-03T08:36:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.