Deep Efficient Private Neighbor Generation for Subgraph Federated Learning

• URL: http://arxiv.org/abs/2401.04336v3
• Date: Fri, 19 Jan 2024 01:30:04 GMT
• Title: Deep Efficient Private Neighbor Generation for Subgraph Federated Learning
• Authors: Ke Zhang, Lichao Sun, Bolin Ding, Siu Ming Yiu, Carl Yang
• Abstract summary: We propose FedDEP to tackle the challenge of incomplete information propagation on local subgraphs due to missing cross-subgraph neighbors. FedDEP consists of a series of novel technical designs: (1) Deep neighbor generation through leveraging the GNN embeddings of potential missing neighbors; (2) Efficient pseudo-FL for neighbor generation through embedding prototyping; and (3) Privacy protection through noise-less edge-local-differential-privacy.
• Score: 57.39918843245229
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Behemoth graphs are often fragmented and separately stored by multiple data owners as distributed subgraphs in many realistic applications. Without harming data privacy, it is natural to consider the subgraph federated learning (subgraph FL) scenario, where each local client holds a subgraph of the entire global graph, to obtain globally generalized graph mining models. To overcome the unique challenge of incomplete information propagation on local subgraphs due to missing cross-subgraph neighbors, previous works resort to the augmentation of local neighborhoods through the joint FL of missing neighbor generators and GNNs. Yet their technical designs have profound limitations regarding the utility, efficiency, and privacy goals of FL. In this work, we propose FedDEP to comprehensively tackle these challenges in subgraph FL. FedDEP consists of a series of novel technical designs: (1) Deep neighbor generation through leveraging the GNN embeddings of potential missing neighbors; (2) Efficient pseudo-FL for neighbor generation through embedding prototyping; and (3) Privacy protection through noise-less edge-local-differential-privacy. We analyze the correctness and efficiency of FedDEP, and provide theoretical guarantees on its privacy. Empirical results on four real-world datasets justify the clear benefits of proposed techniques.

Related papers

• FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction [47.336599393600046]
  textscFedNE is a novel approach that integrates the textscFedAvg framework with the contrastive NE technique. We conduct comprehensive experiments on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-09-17T19:23:24Z)
• FedGT: Federated Node Classification with Scalable Graph Transformer [27.50698154862779]
  We propose a scalable textbfFederated textbfGraph textbfTransformer (textbfFedGT) in the paper. FedGT computes clients' similarity based on the aligned global nodes with optimal transport.
  arXiv  Detail & Related papers  (2024-01-26T21:02:36Z)
• Chasing Fairness in Graphs: A GNN Architecture Perspective [73.43111851492593]
  We propose textsfFair textsfMessage textsfPassing (FMP) designed within a unified optimization framework for graph neural networks (GNNs) In FMP, the aggregation is first adopted to utilize neighbors' information and then the bias mitigation step explicitly pushes demographic group node presentation centers together. Experiments on node classification tasks demonstrate that the proposed FMP outperforms several baselines in terms of fairness and accuracy on three real-world datasets.
  arXiv  Detail & Related papers  (2023-12-19T18:00:15Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Decentralized Graph Neural Network for Privacy-Preserving Recommendation [21.37022040905403]
  This paper proposes DGREC, a novel decentralized GNN for privacy-preserving recommendations. It includes three stages, i.e., graph construction, local gradient calculation, and global gradient passing. We conduct extensive experiments on three public datasets to validate the consistent superiority of our framework.
  arXiv  Detail & Related papers  (2023-08-15T23:56:44Z)
• FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations [53.268801169075836]
  We propose FedLAP-DP, a novel privacy-preserving approach for federated learning. A formal privacy analysis demonstrates that FedLAP-DP incurs the same privacy costs as typical gradient-sharing schemes. Our approach presents a faster convergence speed compared to typical gradient-sharing methods.
  arXiv  Detail & Related papers  (2023-02-02T12:56:46Z)
• Personalized Subgraph Federated Learning [56.52903162729729]
  We introduce a new subgraph FL problem, personalized subgraph FL, which focuses on the joint improvement of the interrelated local GNNs. We propose a novel framework, FEDerated Personalized sUBgraph learning (FED-PUB), to tackle it. We validate our FED-PUB for its subgraph FL performance on six datasets, considering both non-overlapping and overlapping subgraphs.
  arXiv  Detail & Related papers  (2022-06-21T09:02:53Z)
• Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning [86.59588262014456]
  Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. We propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG) Our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
  arXiv  Detail & Related papers  (2022-03-17T11:18:17Z)
• Secure Deep Graph Generation with Link Differential Privacy [32.671503863933616]
  We leverage the differential privacy (DP) framework to formulate and enforce rigorous privacy constraints on deep graph generation models. In particular, we enforce edge-DP by injecting proper noise to the gradients of a link reconstruction-based graph generation model. Our proposed DPGGAN model is able to generate graphs with effectively preserved global structure and rigorously protected individual link privacy.
  arXiv  Detail & Related papers  (2020-05-01T15:49:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.