When the Curious Abandon Honesty: Federated Learning Is Not Private

• URL: http://arxiv.org/abs/2112.02918v2
• Date: Wed, 12 Apr 2023 21:27:15 GMT
• Title: When the Curious Abandon Honesty: Federated Learning Is Not Private
• Authors: Franziska Boenisch, Adam Dziedzic, Roei Schuster, Ali Shahin Shamsabadi, Ilia Shumailov, Nicolas Papernot
• Abstract summary: In federated learning (FL), data does not leave personal devices when they are jointly training a machine learning model. We show a novel data reconstruction attack which allows an active and dishonest central party to efficiently extract user data from the received gradients.
• Score: 36.95590214441999
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In federated learning (FL), data does not leave personal devices when they are jointly training a machine learning model. Instead, these devices share gradients, parameters, or other model updates, with a central party (e.g., a company) coordinating the training. Because data never "leaves" personal devices, FL is often presented as privacy-preserving. Yet, recently it was shown that this protection is but a thin facade, as even a passive, honest-but-curious attacker observing gradients can reconstruct data of individual users contributing to the protocol. In this work, we show a novel data reconstruction attack which allows an active and dishonest central party to efficiently extract user data from the received gradients. While prior work on data reconstruction in FL relies on solving computationally expensive optimization problems or on making easily detectable modifications to the shared model's architecture or parameters, in our attack the central party makes inconspicuous changes to the shared model's weights before sending them out to the users. We call the modified weights of our attack trap weights. Our active attacker is able to recover user data perfectly, i.e., with zero error, even when this data stems from the same class. Recovery comes with near-zero costs: the attack requires no complex optimization objectives. Instead, our attacker exploits inherent data leakage from model gradients and simply amplifies this effect by maliciously altering the weights of the shared model through the trap weights. These specificities enable our attack to scale to fully-connected and convolutional deep neural networks trained with large mini-batches of data. For example, for the high-dimensional vision dataset ImageNet, we perfectly reconstruct more than 50% of the training data points from mini-batches as large as 100 data points.

Related papers

• Leak and Learn: An Attacker's Cookbook to Train Using Leaked Data from Federated Learning [4.533760678036969]
  Federated learning is a decentralized learning paradigm introduced to preserve privacy of client data. Prior work has shown that an attacker can still reconstruct the private training data using only the client updates. We explore data reconstruction attacks through the lens of training and improve models with leaked data.
  arXiv  Detail & Related papers  (2024-03-26T23:05:24Z)
• Client-side Gradient Inversion Against Federated Learning from Poisoning [59.74484221875662]
  Federated Learning (FL) enables distributed participants to train a global model without sharing data directly to a central server. Recent studies have revealed that FL is vulnerable to gradient inversion attack (GIA), which aims to reconstruct the original training samples. We propose Client-side poisoning Gradient Inversion (CGI), which is a novel attack method that can be launched from clients.
  arXiv  Detail & Related papers  (2023-09-14T03:48:27Z)
• Mitigating Adversarial Attacks in Federated Learning with Trusted Execution Environments [1.8240624028534085]
  In image-based applications, adversarial examples consist of images slightly perturbed to the human eye getting misclassified by the local model. Pelta is a novel shielding mechanism leveraging Trusted Execution Environments (TEEs) that reduce the ability of attackers to craft adversarial samples. We show the effectiveness of Pelta in mitigating six white-box state-of-the-art adversarial attacks.
  arXiv  Detail & Related papers  (2023-09-13T14:19:29Z)
• LOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulation [25.03733882637947]
  We introduce LOKI, an attack that overcomes previous limitations and also breaks the anonymity of aggregation. With FedAVG and aggregation across 100 clients, prior work can leak less than 1% of images on MNIST, CIFAR-100, and Tiny ImageNet. Using only a single training round, LOKI is able to leak 76-86% of all data samples.
  arXiv  Detail & Related papers  (2023-03-21T23:29:35Z)
• FL-Defender: Combating Targeted Attacks in Federated Learning [7.152674461313707]
  Federated learning (FL) enables learning a global machine learning model from local data distributed among a set of participating workers. FL is vulnerable to targeted poisoning attacks that negatively impact the integrity of the learned model. We propose textitFL-Defender as a method to combat FL targeted attacks.
  arXiv  Detail & Related papers  (2022-07-02T16:04:46Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)
• Reconstructing Training Data with Informed Adversaries [30.138217209991826]
  Given access to a machine learning model, can an adversary reconstruct the model's training data? This work studies this question from the lens of a powerful informed adversary who knows all the training data points except one. We show it is feasible to reconstruct the remaining data point in this stringent threat model.
  arXiv  Detail & Related papers  (2022-01-13T09:19:25Z)
• Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models [56.0250919557652]
  Federated learning has quickly gained popularity with its promises of increased user privacy and efficiency. Previous attacks on user privacy have been limited in scope and do not scale to gradient updates aggregated over even a handful of data points. We introduce a new threat model based on minimal but malicious modifications of the shared model architecture.
  arXiv  Detail & Related papers  (2021-10-25T15:52:06Z)
• Knowledge-Enriched Distributional Model Inversion Attacks [49.43828150561947]
  Model inversion (MI) attacks are aimed at reconstructing training data from model parameters. We present a novel inversion-specific GAN that can better distill knowledge useful for performing attacks on private models from public data. Our experiments show that the combination of these techniques can significantly boost the success rate of the state-of-the-art MI attacks by 150%.
  arXiv  Detail & Related papers  (2020-10-08T16:20:48Z)
• How Does Data Augmentation Affect Privacy in Machine Learning? [94.52721115660626]
  We propose new MI attacks to utilize the information of augmented data. We establish the optimal membership inference when the model is trained with augmented data.
  arXiv  Detail & Related papers  (2020-07-21T02:21:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.