Private and Communication-Efficient Federated Learning based on Differentially Private Sketches

• URL: http://arxiv.org/abs/2410.05733v2
• Date: Thu, 10 Oct 2024 03:35:54 GMT
• Title: Private and Communication-Efficient Federated Learning based on Differentially Private Sketches
• Authors: Meifan Zhang, Zhanhong Xie, Lihua Yin,
• Abstract summary: Federated learning (FL) faces two primary challenges: the risk of privacy leakage and communication inefficiencies. We propose DPSFL, a federated learning method that utilizes differentially private sketches. We provide a theoretical analysis of privacy and convergence for the proposed method.
• Score: 0.4533408985664949
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) faces two primary challenges: the risk of privacy leakage due to parameter sharing and communication inefficiencies. To address these challenges, we propose DPSFL, a federated learning method that utilizes differentially private sketches. DPSFL compresses the local gradients of each client using a count sketch, thereby improving communication efficiency, while adding noise to the sketches to ensure differential privacy (DP). We provide a theoretical analysis of privacy and convergence for the proposed method. Gradient clipping is essential in DP learning to limit sensitivity and constrain the addition of noise. However, clipping introduces bias into the gradients, negatively impacting FL performance. To mitigate the impact of clipping, we propose an enhanced method, DPSFL-AC, which employs an adaptive clipping strategy. Experimental comparisons with existing techniques demonstrate the superiority of our methods concerning privacy preservation, communication efficiency, and model accuracy.

Related papers

• DP$^2$-FedSAM: Enhancing Differentially Private Federated Learning Through Personalized Sharpness-Aware Minimization [8.022417295372492]
  Federated learning (FL) is a distributed machine learning approach that allows multiple clients to collaboratively train a model without sharing their raw data. To prevent sensitive information from being inferred through the model updates shared in FL, differentially private federated learning (DPFL) has been proposed. DPFL ensures formal and rigorous privacy protection in FL by clipping and adding random noise to the shared model updates. We propose DP$2$-FedSAM: Differentially Private and Personalized Federated Learning with Sharpness-Aware Minimization.
  arXiv  Detail & Related papers  (2024-09-20T16:49:01Z)
• Personalized Federated Learning Techniques: Empirical Analysis [2.9521571597754885]
  We empirically evaluate ten prominent pFL techniques across various datasets and data splits, uncovering significant differences in their performance. Our study emphasizes the critical role of communication efficiency in scaling pFL, demonstrating how it can significantly affect resource usage in real-world deployments.
  arXiv  Detail & Related papers  (2024-09-10T18:16:28Z)
• Binary Federated Learning with Client-Level Differential Privacy [7.854806519515342]
  Federated learning (FL) is a privacy-preserving collaborative learning framework. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm. We propose a communication-efficient FL training algorithm with differential privacy guarantee.
  arXiv  Detail & Related papers  (2023-08-07T06:07:04Z)
• Differentially Private Wireless Federated Learning Using Orthogonal Sequences [56.52483669820023]
  We propose a privacy-preserving uplink over-the-air computation (AirComp) method, termed FLORAS. We prove that FLORAS offers both item-level and client-level differential privacy guarantees. A new FL convergence bound is derived which, combined with the privacy guarantees, allows for a smooth tradeoff between the achieved convergence rate and differential privacy levels.
  arXiv  Detail & Related papers  (2023-06-14T06:35:10Z)
• FedLAP-DP: Federated Learning by Sharing Differentially Private Loss Approximations [53.268801169075836]
  We propose FedLAP-DP, a novel privacy-preserving approach for federated learning. A formal privacy analysis demonstrates that FedLAP-DP incurs the same privacy costs as typical gradient-sharing schemes. Our approach presents a faster convergence speed compared to typical gradient-sharing methods.
  arXiv  Detail & Related papers  (2023-02-02T12:56:46Z)
• Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise [30.005017338416327]
  Federated learning seeks to address the issue of isolated data islands by making clients disclose only their local training models. Recently, differential privacy has been applied to federated learning to protect data privacy, but the noise added may degrade the learning performance much. We propose a differentially private scheme for federated learning with adaptive noise (Adap DP-FL)
  arXiv  Detail & Related papers  (2022-11-29T03:20:40Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)
• Voting-based Approaches For Differentially Private Federated Learning [87.2255217230752]
  This work is inspired by knowledge transfer non-federated privacy learning from Papernot et al. We design two new DPFL schemes, by voting among the data labels returned from each local model, instead of averaging the gradients. Our approaches significantly improve the privacy-utility trade-off over the state-of-the-arts in DPFL.
  arXiv  Detail & Related papers  (2020-10-09T23:55:19Z)
• Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization [27.243322019117144]
  Federated learning (FL) enables distributed agents to collaboratively learn a centralized model without sharing their raw data with each other. We propose a new FL framework with sparsification-amplified privacy. Our approach integrates random sparsification with gradient perturbation on each agent to amplify privacy guarantee.
  arXiv  Detail & Related papers  (2020-08-01T20:22:57Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.