Binary Federated Learning with Client-Level Differential Privacy

• URL: http://arxiv.org/abs/2308.03320v1
• Date: Mon, 7 Aug 2023 06:07:04 GMT
• Title: Binary Federated Learning with Client-Level Differential Privacy
• Authors: Lumin Liu, Jun Zhang, Shenghui Song, Khaled B. Letaief
• Abstract summary: Federated learning (FL) is a privacy-preserving collaborative learning framework. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm. We propose a communication-efficient FL training algorithm with differential privacy guarantee.
• Score: 7.854806519515342
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) is a privacy-preserving collaborative learning framework, and differential privacy can be applied to further enhance its privacy protection. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm and implement differential privacy with a Gaussian mechanism. However, the inherent privacy-utility trade-off in these systems severely degrades the training performance if a tight privacy budget is enforced. Besides, the Gaussian mechanism requires model weights to be of high-precision. To improve communication efficiency and achieve a better privacy-utility trade-off, we propose a communication-efficient FL training algorithm with differential privacy guarantee. Specifically, we propose to adopt binary neural networks (BNNs) and introduce discrete noise in the FL setting. Binary model parameters are uploaded for higher communication efficiency and discrete noise is added to achieve the client-level differential privacy protection. The achieved performance guarantee is rigorously proved, and it is shown to depend on the level of discrete noise. Experimental results based on MNIST and Fashion-MNIST datasets will demonstrate that the proposed training algorithm achieves client-level privacy protection with performance gain while enjoying the benefits of low communication overhead from binary model updates.

Related papers

• Adaptive Client Selection in Federated Learning: A Network Anomaly Detection Use Case [0.30723404270319693]
  This paper introduces a client selection framework for Federated Learning (FL) that incorporates differential privacy and fault tolerance. Results demonstrate up to a 7% improvement in accuracy and a 25% reduction in training time compared to the FedL2P approach.
  arXiv  Detail & Related papers  (2025-01-25T02:50:46Z)
• Communication-Efficient and Privacy-Adaptable Mechanism for Federated Learning [33.267664801299354]
  Training machine learning models on decentralized private data via federated learning (FL) poses two key challenges: communication efficiency and privacy protection. We introduce a novel approach called the Communication-Efficient and Privacy-Adaptable Mechanism (CEPAM), achieving both objectives simultaneously. We analyze the trade-offs among user privacy, global utility, and transmission rate of CEPAM by defining appropriate metrics for FL with differential privacy and compression.
  arXiv  Detail & Related papers  (2025-01-21T11:16:05Z)
• Providing Differential Privacy for Federated Learning Over Wireless: A Cross-layer Framework [19.381425127772054]
  Federated Learning (FL) is a distributed machine learning framework that inherently allows edge devices to maintain their local training data. We propose a wireless physical layer (PHY) design for OTA-FL which improves differential privacy (DP) through a decentralized, dynamic power control. This adaptation showcases the flexibility and effectiveness of our design across different learning algorithms while maintaining a strong emphasis on privacy.
  arXiv  Detail & Related papers  (2024-12-05T18:27:09Z)
• CorBin-FL: A Differentially Private Federated Learning Mechanism using Common Randomness [6.881974834597426]
  Federated learning (FL) has emerged as a promising framework for distributed machine learning. We introduce CorBin-FL, a privacy mechanism that uses correlated binary quantization to achieve differential privacy. We also propose AugCorBin-FL, an extension that, in addition to PLDP, user-level and sample-level central differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-09-20T00:23:44Z)
• Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification [51.04894019092156]
  Federated learning (FL) has been recognized as a rapidly growing area, where the model is trained over clients under the FL orchestration (PS) In this paper, we propose a novel primal sparification algorithm for and guarantee non-smooth FL problems. Its unique insightful properties and its analyses are also presented.
  arXiv  Detail & Related papers  (2023-10-30T14:15:47Z)
• Differentially Private Wireless Federated Learning Using Orthogonal Sequences [56.52483669820023]
  We propose a privacy-preserving uplink over-the-air computation (AirComp) method, termed FLORAS. We prove that FLORAS offers both item-level and client-level differential privacy guarantees. A new FL convergence bound is derived which, combined with the privacy guarantees, allows for a smooth tradeoff between the achieved convergence rate and differential privacy levels.
  arXiv  Detail & Related papers  (2023-06-14T06:35:10Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• Over-the-Air Federated Learning with Privacy Protection via Correlated Additive Perturbations [57.20885629270732]
  We consider privacy aspects of wireless federated learning with Over-the-Air (OtA) transmission of gradient updates from multiple users/agents to an edge server. Traditional perturbation-based methods provide privacy protection while sacrificing the training accuracy. In this work, we aim at minimizing privacy leakage to the adversary and the degradation of model accuracy at the edge server.
  arXiv  Detail & Related papers  (2022-10-05T13:13:35Z)
• Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy [67.4471689755097]
  This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity. We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
  arXiv  Detail & Related papers  (2021-06-25T14:47:19Z)
• Federated Learning with Sparsification-Amplified Privacy and Adaptive Optimization [27.243322019117144]
  Federated learning (FL) enables distributed agents to collaboratively learn a centralized model without sharing their raw data with each other. We propose a new FL framework with sparsification-amplified privacy. Our approach integrates random sparsification with gradient perturbation on each agent to amplify privacy guarantee.
  arXiv  Detail & Related papers  (2020-08-01T20:22:57Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.