Differentially Private Federated Learning without Noise Addition: When is it Possible?

• URL: http://arxiv.org/abs/2405.04551v2
• Date: Tue, 4 Jun 2024 05:17:56 GMT
• Title: Differentially Private Federated Learning without Noise Addition: When is it Possible?
• Authors: Jiang Zhang, Konstantinos Psounis,
• Abstract summary: Federated Learning with Secure Aggregation (SA) has gained significant attention as a privacy preserving framework for training machine learning models. Recent research has extended privacy guarantees of FL with SA by bounding the information leakage through the aggregate model over multiple training rounds. We study the conditions under which FL with SA can provide worst-case differential privacy guarantees.
• Score: 5.2643054813504175
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Federated Learning (FL) with Secure Aggregation (SA) has gained significant attention as a privacy preserving framework for training machine learning models while preventing the server from learning information about users' data from their individual encrypted model updates. Recent research has extended privacy guarantees of FL with SA by bounding the information leakage through the aggregate model over multiple training rounds thanks to leveraging the "noise" from other users' updates. However, the privacy metric used in that work (mutual information) measures the on-average privacy leakage, without providing any privacy guarantees for worse-case scenarios. To address this, in this work we study the conditions under which FL with SA can provide worst-case differential privacy guarantees. Specifically, we formally identify the necessary condition that SA can provide DP without addition noise. We then prove that when the randomness inside the aggregated model update is Gaussian with non-singular covariance matrix, SA can provide differential privacy guarantees with the level of privacy $\epsilon$ bounded by the reciprocal of the minimum eigenvalue of the covariance matrix. However, we further demonstrate that in practice, these conditions are almost unlikely to hold and hence additional noise added in model updates is still required in order for SA in FL to achieve DP. Lastly, we discuss the potential solution of leveraging inherent randomness inside aggregated model update to reduce the amount of addition noise required for DP guarantee.

Related papers

• PriRoAgg: Achieving Robust Model Aggregation with Minimum Privacy Leakage for Federated Learning [49.916365792036636]
  Federated learning (FL) has recently gained significant momentum due to its potential to leverage large-scale distributed user data. The transmitted model updates can potentially leak sensitive user information, and the lack of central control of the local training process leaves the global model susceptible to malicious manipulations on model updates. We develop a general framework PriRoAgg, utilizing Lagrange coded computing and distributed zero-knowledge proof, to execute a wide range of robust aggregation algorithms while satisfying aggregated privacy.
  arXiv  Detail & Related papers  (2024-07-12T03:18:08Z)
• Share Your Representation Only: Guaranteed Improvement of the Privacy-Utility Tradeoff in Federated Learning [47.042811490685324]
  Mitigating the risk of this information leakage, using state of the art differentially private algorithms, also does not come for free. In this paper, we consider a representation learning objective that various parties collaboratively refine on a federated model, with differential privacy guarantees. We observe a significant performance improvement over the prior work under the same small privacy budget.
  arXiv  Detail & Related papers  (2023-09-11T14:46:55Z)
• Probing the Transition to Dataset-Level Privacy in ML Models Using an Output-Specific and Data-Resolved Privacy Profile [23.05994842923702]
  We study a privacy metric that quantifies the extent to which a model trained on a dataset using a Differential Privacy mechanism is covered" by each of the distributions resulting from training on neighboring datasets. We show that the privacy profile can be used to probe an observed transition to indistinguishability that takes place in the neighboring distributions as $epsilon$ decreases.
  arXiv  Detail & Related papers  (2023-06-27T20:39:07Z)
• A Randomized Approach for Tight Privacy Accounting [63.67296945525791]
  We propose a new differential privacy paradigm called estimate-verify-release (EVR) EVR paradigm first estimates the privacy parameter of a mechanism, then verifies whether it meets this guarantee, and finally releases the query output. Our empirical evaluation shows the newly proposed EVR paradigm improves the utility-privacy tradeoff for privacy-preserving machine learning.
  arXiv  Detail & Related papers  (2023-04-17T00:38:01Z)
• Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning [86.08285033925597]
  This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of federated learning. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.
  arXiv  Detail & Related papers  (2023-03-07T22:52:40Z)
• Privacy-Preserving Distributed Expectation Maximization for Gaussian Mixture Model using Subspace Perturbation [4.2698418800007865]
  federated learning is motivated by the privacy concern as it does not allow to transmit private data but only intermediate updates. We propose a fully decentralized privacy-preserving solution, which is able to securely compute the updates in each step. Numerical validation shows that the proposed approach has superior performance compared to the existing approach in terms of both the accuracy and privacy level.
  arXiv  Detail & Related papers  (2022-09-16T09:58:03Z)
• How Much Privacy Does Federated Learning with Secure Aggregation Guarantee? [22.7443077369789]
  Federated learning (FL) has attracted growing interest for enabling privacy-preserving machine learning on data stored at multiple users. While data never leaves users' devices, privacy still cannot be guaranteed since significant computations on users' training data are shared in the form of trained local models. Secure Aggregation (SA) has been developed as a framework to preserve privacy in FL.
  arXiv  Detail & Related papers  (2022-08-03T18:44:17Z)
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
  We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-06T13:49:37Z)
• A Shuffling Framework for Local Differential Privacy [40.92785300658643]
  ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity. An alternative model, shuffle DP, prevents this by shuffling the noisy responses uniformly at random. We show that systematic shuffling of the noisy responses can thwart specific inference attacks while retaining some meaningful data learnability.
  arXiv  Detail & Related papers  (2021-06-11T20:36:23Z)
• Privacy Amplification via Random Check-Ins [38.72327434015975]
  Differentially Private Gradient Descent (DP-SGD) forms a fundamental building block in many applications for learning over sensitive data. In this paper, we focus on conducting iterative methods like DP-SGD in the setting of federated learning (FL) wherein the data is distributed among many devices (clients) Our main contribution is the emphrandom check-in distributed protocol, which crucially relies only on randomized participation decisions made locally and independently by each client.
  arXiv  Detail & Related papers  (2020-07-13T18:14:09Z)
• Differentially Private Federated Learning with Laplacian Smoothing [72.85272874099644]
  Federated learning aims to protect data privacy by collaboratively learning a model without sharing private data among users. An adversary may still be able to infer the private training data by attacking the released model. Differential privacy provides a statistical protection against such attacks at the price of significantly degrading the accuracy or utility of the trained models.
  arXiv  Detail & Related papers  (2020-05-01T04:28:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.