Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning

• URL: http://arxiv.org/abs/2304.13545v1
• Date: Wed, 26 Apr 2023 13:13:04 GMT
• Title: Killing Two Birds with One Stone: Quantization Achieves Privacy in Distributed Learning
• Authors: Guangfeng Yan, Tan Li, Kui Wu, Linqi Song
• Abstract summary: Communication efficiency and privacy protection are critical issues in distributed machine learning. We propose a comprehensive quantization-based solution that could simultaneously achieve communication efficiency and privacy protection. We theoretically capture the new trade-offs between communication, privacy, and learning performance.
• Score: 18.824571167583432
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Communication efficiency and privacy protection are two critical issues in distributed machine learning. Existing methods tackle these two issues separately and may have a high implementation complexity that constrains their application in a resource-limited environment. We propose a comprehensive quantization-based solution that could simultaneously achieve communication efficiency and privacy protection, providing new insights into the correlated nature of communication and privacy. Specifically, we demonstrate the effectiveness of our proposed solutions in the distributed stochastic gradient descent (SGD) framework by adding binomial noise to the uniformly quantized gradients to reach the desired differential privacy level but with a minor sacrifice in communication efficiency. We theoretically capture the new trade-offs between communication, privacy, and learning performance.

Related papers

• Collaborative Inference over Wireless Channels with Feature Differential Privacy [57.68286389879283]
  Collaborative inference among multiple wireless edge devices has the potential to significantly enhance Artificial Intelligence (AI) applications. transmitting extracted features poses a significant privacy risk, as sensitive personal data can be exposed during the process. We propose a novel privacy-preserving collaborative inference mechanism, wherein each edge device in the network secures the privacy of extracted features before transmitting them to a central server for inference.
  arXiv  Detail & Related papers  (2024-10-25T18:11:02Z)
• Federated Cubic Regularized Newton Learning with Sparsification-amplified Differential Privacy [10.396575601912673]
  We introduce a federated learning algorithm called Differentially Private Federated Cubic Regularized Newton (DP-FCRN) By leveraging second-order techniques, our algorithm achieves lower iteration complexity compared to first-order methods. We also incorporate noise perturbation during local computations to ensure privacy.
  arXiv  Detail & Related papers  (2024-08-08T08:48:54Z)
• TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data [50.797729676285876]
  We propose TernaryVote, which combines a ternary compressor and the majority vote mechanism to realize differential privacy, gradient compression, and Byzantine resilience simultaneously. We theoretically quantify the privacy guarantee through the lens of the emerging f-differential privacy (DP) and the Byzantine resilience of the proposed algorithm.
  arXiv  Detail & Related papers  (2024-02-16T16:41:14Z)
• Binary Federated Learning with Client-Level Differential Privacy [7.854806519515342]
  Federated learning (FL) is a privacy-preserving collaborative learning framework. Existing FL systems typically adopt Federated Average (FedAvg) as the training algorithm. We propose a communication-efficient FL training algorithm with differential privacy guarantee.
  arXiv  Detail & Related papers  (2023-08-07T06:07:04Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy [51.11280118806893]
  We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. We study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP) More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms.
  arXiv  Detail & Related papers  (2023-02-19T16:58:53Z)
• Privacy-Preserving Joint Edge Association and Power Optimization for the Internet of Vehicles via Federated Multi-Agent Reinforcement Learning [74.53077322713548]
  We investigate the privacy-preserving joint edge association and power allocation problem. The proposed solution strikes a compelling trade-off, while preserving a higher privacy level than the state-of-the-art solutions.
  arXiv  Detail & Related papers  (2023-01-26T10:09:23Z)
• SoteriaFL: A Unified Framework for Private Federated Learning with Communication Compression [40.646108010388986]
  We propose a unified framework that enhances the communication efficiency of private federated learning with communication compression. We provide a comprehensive characterization of its performance trade-offs in terms of privacy, utility, and communication complexity.
  arXiv  Detail & Related papers  (2022-06-20T16:47:58Z)
• Differentially Private Multi-Agent Planning for Logistic-like Problems [70.3758644421664]
  This paper proposes a novel strong privacy-preserving planning approach for logistic-like problems. Two challenges are addressed: 1) simultaneously achieving strong privacy, completeness and efficiency, and 2) addressing communication constraints. To the best of our knowledge, this paper is the first to apply differential privacy to the field of multi-agent planning.
  arXiv  Detail & Related papers  (2020-08-16T03:43:09Z)
• FedSKETCH: Communication-Efficient and Private Federated Learning via Sketching [33.54413645276686]
  Communication complexity and privacy are the two key challenges in Federated Learning. We introduce FedSKETCH and FedSKETCHGATE algorithms to address both challenges in Federated learning jointly.
  arXiv  Detail & Related papers  (2020-08-11T19:22:48Z)
• Differentially private cross-silo federated learning [16.38610531397378]
  Strict privacy is of paramount importance in distributed machine learning. In this paper we combine additively homomorphic secure summation protocols with differential privacy in the so-called cross-silo federated learning setting. We demonstrate that our proposed solutions give prediction accuracy that is comparable to the non-distributed setting.
  arXiv  Detail & Related papers  (2020-07-10T18:15:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.