Secure Aggregation is Not Private Against Membership Inference Attacks

• URL: http://arxiv.org/abs/2403.17775v3
• Date: Mon, 15 Jul 2024 14:29:33 GMT
• Title: Secure Aggregation is Not Private Against Membership Inference Attacks
• Authors: Khac-Hoang Ngo, Johan Östman, Giuseppe Durisi, Alexandre Graell i Amat,
• Abstract summary: We investigate the privacy implications of SecAgg in federated learning. We show that SecAgg offers weak privacy against membership inference attacks even in a single training round. Our findings underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection.
• Score: 66.59892736942953
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Secure aggregation (SecAgg) is a commonly-used privacy-enhancing mechanism in federated learning, affording the server access only to the aggregate of model updates while safeguarding the confidentiality of individual updates. Despite widespread claims regarding SecAgg's privacy-preserving capabilities, a formal analysis of its privacy is lacking, making such presumptions unjustified. In this paper, we delve into the privacy implications of SecAgg by treating it as a local differential privacy (LDP) mechanism for each local update. We design a simple attack wherein an adversarial server seeks to discern which update vector a client submitted, out of two possible ones, in a single training round of federated learning under SecAgg. By conducting privacy auditing, we assess the success probability of this attack and quantify the LDP guarantees provided by SecAgg. Our numerical results unveil that, contrary to prevailing claims, SecAgg offers weak privacy against membership inference attacks even in a single training round. Indeed, it is difficult to hide a local update by adding other independent local updates when the updates are of high dimension. Our findings underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection, in federated learning.

Related papers

• Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy [6.343100139647636]
  Federated learning (FL) is designed to preserve data privacy during model training. FL is vulnerable to privacy attacks and Byzantine attacks. We propose a new FL scheme that guarantees rigorous privacy and simultaneously enhances system robustness against Byzantine attacks.
  arXiv  Detail & Related papers  (2023-09-07T01:39:02Z)
• Client-specific Property Inference against Secure Aggregation in Federated Learning [52.8564467292226]
  Federated learning has become a widely used paradigm for collaboratively training a common model among different participants. Many attacks have shown that it is still possible to infer sensitive information such as membership, property, or outright reconstruction of participant data. We show that simple linear models can effectively capture client-specific properties only from the aggregated model updates.
  arXiv  Detail & Related papers  (2023-03-07T14:11:01Z)
• Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy [51.11280118806893]
  We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. We study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP) More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms.
  arXiv  Detail & Related papers  (2023-02-19T16:58:53Z)
• Privacy-Preserving Distributed Expectation Maximization for Gaussian Mixture Model using Subspace Perturbation [4.2698418800007865]
  federated learning is motivated by the privacy concern as it does not allow to transmit private data but only intermediate updates. We propose a fully decentralized privacy-preserving solution, which is able to securely compute the updates in each step. Numerical validation shows that the proposed approach has superior performance compared to the existing approach in terms of both the accuracy and privacy level.
  arXiv  Detail & Related papers  (2022-09-16T09:58:03Z)
• A New Implementation of Federated Learning for Privacy and Security Enhancement [27.612480082254486]
  Federated learning (FL) has emerged as a new machine learning setting. No local data needs to be shared, and privacy can be well protected. We propose a model update based federated averaging algorithm to defend against Byzantine attacks.
  arXiv  Detail & Related papers  (2022-08-03T03:13:19Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)
• Defending against Reconstruction Attacks with R\'enyi Differential Privacy [72.1188520352079]
  Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget. We show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature.
  arXiv  Detail & Related papers  (2022-02-15T18:09:30Z)
• Secure and Privacy-Preserving Federated Learning via Co-Utility [7.428782604099875]
  We build a federated learning framework that offers privacy to the participating peers and security against Byzantine and poisoning attacks. Unlike privacy protection via update aggregation, our approach preserves the values of model updates and hence the accuracy of plain federated learning.
  arXiv  Detail & Related papers  (2021-08-04T08:58:24Z)
• Federated Learning in Adversarial Settings [0.8701566919381224]
  Federated learning scheme provides different trade-offs between robustness, privacy, bandwidth efficiency, and model accuracy. We show that this extension performs as efficiently as the non-private but robust scheme, even with stringent privacy requirements. This suggests a possible fundamental trade-off between Differential Privacy and robustness.
  arXiv  Detail & Related papers  (2020-10-15T14:57:02Z)
• Private Reinforcement Learning with PAC and Regret Guarantees [69.4202374491817]
  We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
  arXiv  Detail & Related papers  (2020-09-18T20:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.