Privacy-Aware Compression for Federated Learning Through Numerical
Mechanism Design
- URL: http://arxiv.org/abs/2211.03942v3
- Date: Thu, 10 Aug 2023 02:55:51 GMT
- Title: Privacy-Aware Compression for Federated Learning Through Numerical
Mechanism Design
- Authors: Chuan Guo, Kamalika Chaudhuri, Pierre Stock, Mike Rabbat
- Abstract summary: This paper introduces a new procedure in the numerical design process that allows for a far more efficient privacy analysis.
The Interpolated MVU mechanism is more scalable, has a better privacy-utility trade-off, and provides SOTA results on communication-efficient private FL on a variety of datasets.
- Score: 32.45650219508591
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In private federated learning (FL), a server aggregates differentially
private updates from a large number of clients in order to train a machine
learning model. The main challenge in this setting is balancing privacy with
both classification accuracy of the learnt model as well as the number of bits
communicated between the clients and server. Prior work has achieved a good
trade-off by designing a privacy-aware compression mechanism, called the
minimum variance unbiased (MVU) mechanism, that numerically solves an
optimization problem to determine the parameters of the mechanism. This paper
builds upon it by introducing a new interpolation procedure in the numerical
design process that allows for a far more efficient privacy analysis. The
result is the new Interpolated MVU mechanism that is more scalable, has a
better privacy-utility trade-off, and provides SOTA results on
communication-efficient private FL on a variety of datasets.
Related papers
- Clients Collaborate: Flexible Differentially Private Federated Learning
with Guaranteed Improvement of Utility-Privacy Trade-off [34.2117116062642]
We introduce a novel federated learning framework with rigorous privacy guarantees, named FedCEO, to strike a trade-off between model utility and user privacy.
We show that our FedCEO can effectively recover the disrupted semantic information by smoothing the global semantic space.
It observes significant performance improvements and strict privacy guarantees under different privacy settings.
arXiv Detail & Related papers (2024-02-10T17:39:34Z) - Federated Learning with Differential Privacy [0.9208007322096533]
Federated learning (FL) is capable of preserving client's private data from being shared among different parties.
Private information can still be divulged by analyzing uploaded parameter weights from clients.
Our results show that non-i.i.d and small datasets have the highest decrease in performance in a distributed and differentially private setting.
arXiv Detail & Related papers (2024-02-03T18:21:38Z) - QMGeo: Differentially Private Federated Learning via Stochastic Quantization with Mixed Truncated Geometric Distribution [1.565361244756411]
Federated learning (FL) is a framework which allows multiple users to jointly train a global machine learning (ML) model.
One key motivation of such distributed frameworks is to provide privacy guarantees to the users.
We present a novel quantization method, utilizing a mixed geometric distribution to introduce the randomness needed to provide DP.
arXiv Detail & Related papers (2023-12-10T04:44:53Z) - Differentially Private Over-the-Air Federated Learning Over MIMO Fading
Channels [24.534729104570417]
Federated learning (FL) enables edge devices to collaboratively train machine learning models.
While over-the-air model aggregation improves communication efficiency, uploading models to an edge server over wireless networks can pose privacy risks.
We show that FL model communication with a multiple-antenna server amplifies privacy leakage.
arXiv Detail & Related papers (2023-06-19T14:44:34Z) - Balancing Privacy and Performance for Private Federated Learning
Algorithms [4.681076651230371]
Federated learning (FL) is a distributed machine learning framework where multiple clients collaborate to train a model without exposing their private data.
FL algorithms frequently employ a differential privacy mechanism that introduces noise into each client's model updates before sharing.
We show that an optimal balance exists between the number of local steps and communication rounds, one that maximizes the convergence performance within a given privacy budget.
arXiv Detail & Related papers (2023-04-11T10:42:11Z) - Federated Nearest Neighbor Machine Translation [66.8765098651988]
In this paper, we propose a novel federated nearest neighbor (FedNN) machine translation framework.
FedNN leverages one-round memorization-based interaction to share knowledge across different clients.
Experiments show that FedNN significantly reduces computational and communication costs compared with FedAvg.
arXiv Detail & Related papers (2023-02-23T18:04:07Z) - On Differential Privacy for Federated Learning in Wireless Systems with
Multiple Base Stations [90.53293906751747]
We consider a federated learning model in a wireless system with multiple base stations and inter-cell interference.
We show the convergence behavior of the learning process by deriving an upper bound on its optimality gap.
Our proposed scheduler improves the average accuracy of the predictions compared with a random scheduler.
arXiv Detail & Related papers (2022-08-25T03:37:11Z) - Personalization Improves Privacy-Accuracy Tradeoffs in Federated
Optimization [57.98426940386627]
We show that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy.
We illustrate our theoretical results with experiments on synthetic and real-world datasets.
arXiv Detail & Related papers (2022-02-10T20:44:44Z) - An Expectation-Maximization Perspective on Federated Learning [75.67515842938299]
Federated learning describes the distributed training of models across multiple clients while keeping the data private on-device.
In this work, we view the server-orchestrated federated learning process as a hierarchical latent variable model where the server provides the parameters of a prior distribution over the client-specific model parameters.
We show that with simple Gaussian priors and a hard version of the well known Expectation-Maximization (EM) algorithm, learning in such a model corresponds to FedAvg, the most popular algorithm for the federated learning setting.
arXiv Detail & Related papers (2021-11-19T12:58:59Z) - Understanding Clipping for Federated Learning: Convergence and
Client-Level Differential Privacy [67.4471689755097]
This paper empirically demonstrates that the clipped FedAvg can perform surprisingly well even with substantial data heterogeneity.
We provide the convergence analysis of a differential private (DP) FedAvg algorithm and highlight the relationship between clipping bias and the distribution of the clients' updates.
arXiv Detail & Related papers (2021-06-25T14:47:19Z) - Federated Learning with Unreliable Clients: Performance Analysis and
Mechanism Design [76.29738151117583]
Federated Learning (FL) has become a promising tool for training effective machine learning models among distributed clients.
However, low quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training.
We model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk.
arXiv Detail & Related papers (2021-05-10T08:02:27Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.