Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis

• URL: http://arxiv.org/abs/2010.13356v2
• Date: Wed, 22 Dec 2021 12:27:19 GMT
• Title: Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis
• Authors: Xudong Pan, Mi Zhang, Yifan Yan, Jiaming Zhu, Min Yang
• Abstract summary: We study the security boundary of data reconstruction from gradient via a microcosmic view on neural networks with rectified linear units (ReLUs) We construct a novel deterministic attack algorithm which substantially outperforms previous attacks for reconstructing training batches lying in the insecure boundary of a neural network.
• Score: 23.07323180340961
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Among existing privacy attacks on the gradient of neural networks, \emph{data reconstruction attack}, which reverse engineers the training batch from the gradient, poses a severe threat on the private training data. Despite its empirical success on large architectures and small training batches, unstable reconstruction accuracy is also observed when a smaller architecture or a larger batch is under attack. Due to the weak interpretability of existing learning-based attacks, there is little known on why, when and how data reconstruction attack is feasible. In our work, we perform the first analytic study on the security boundary of data reconstruction from gradient via a microcosmic view on neural networks with rectified linear units (ReLUs), the most popular activation function in practice. For the first time, we characterize the insecure/secure boundary of data reconstruction attack in terms of the \emph{neuron exclusivity state} of a training batch, indexed by the number of \emph{\textbf{Ex}clusively \textbf{A}ctivated \textbf{N}eurons} (ExANs, i.e., a ReLU activated by only one sample in a batch). Intuitively, we show a training batch with more ExANs are more vulnerable to data reconstruction attack and vice versa. On the one hand, we construct a novel deterministic attack algorithm which substantially outperforms previous attacks for reconstructing training batches lying in the insecure boundary of a neural network. Meanwhile, for training batches lying in the secure boundary, we prove the impossibility of unique reconstruction, based on which an exclusivity reduction strategy is devised to enlarge the secure boundary for mitigation purposes.

Related papers

• Extracting Spatiotemporal Data from Gradients with Large Language Models [30.785476975412482]
  Recent updates that can be updated from gradient data break key privacy promise of federated learning. We propose an adaptive defense strategy to mitigate attacks in federated learning. We show that the proposed defense strategy can well preserve the utility of thetemporal-temporal federated learning with effective security protection.
  arXiv  Detail & Related papers  (2024-10-21T15:48:34Z)
• A Practical Trigger-Free Backdoor Attack on Neural Networks [33.426207982772226]
  We propose a trigger-free backdoor attack that does not require access to any training data. Specifically, we design a novel fine-tuning approach that incorporates the concept of malicious data into the concept of the attacker-specified class. The effectiveness, practicality, and stealthiness of the proposed attack are evaluated on three real-world datasets.
  arXiv  Detail & Related papers  (2024-08-21T08:53:36Z)
• Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning [49.242828934501986]
  Multimodal contrastive learning has emerged as a powerful paradigm for building high-quality features. backdoor attacks subtly embed malicious behaviors within the model during training. We introduce an innovative token-based localized forgetting training regime.
  arXiv  Detail & Related papers  (2024-03-24T18:33:15Z)
• Bounding Reconstruction Attack Success of Adversaries Without Data Priors [53.41619942066895]
  Reconstruction attacks on machine learning (ML) models pose a strong risk of leakage of sensitive data. In this work, we provide formal upper bounds on reconstruction success under realistic adversarial settings.
  arXiv  Detail & Related papers  (2024-02-20T09:52:30Z)
• Understanding Reconstruction Attacks with the Neural Tangent Kernel and Dataset Distillation [110.61853418925219]
  We build a stronger version of the dataset reconstruction attack and show how it can provably recover the emphentire training set in the infinite width regime. We show that both theoretically and empirically, reconstructed images tend to "outliers" in the dataset. These reconstruction attacks can be used for textitdataset distillation, that is, we can retrain on reconstructed images and obtain high predictive accuracy.
  arXiv  Detail & Related papers  (2023-02-02T21:41:59Z)
• Reconstructing Training Data from Model Gradient, Provably [68.21082086264555]
  We reconstruct the training samples from a single gradient query at a randomly chosen parameter value. As a provable attack that reveals sensitive training data, our findings suggest potential severe threats to privacy.
  arXiv  Detail & Related papers  (2022-12-07T15:32:22Z)
• Do Gradient Inversion Attacks Make Federated Learning Unsafe? [70.0231254112197]
  Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. Recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in real FL use-cases and provide a new baseline attack.
  arXiv  Detail & Related papers  (2022-02-14T18:33:12Z)
• Reconstructing Training Data with Informed Adversaries [30.138217209991826]
  Given access to a machine learning model, can an adversary reconstruct the model's training data? This work studies this question from the lens of a powerful informed adversary who knows all the training data points except one. We show it is feasible to reconstruct the remaining data point in this stringent threat model.
  arXiv  Detail & Related papers  (2022-01-13T09:19:25Z)
• Meta Adversarial Perturbations [66.43754467275967]
  We show the existence of a meta adversarial perturbation (MAP) MAP causes natural images to be misclassified with high probability after being updated through only a one-step gradient ascent update. We show that these perturbations are not only image-agnostic, but also model-agnostic, as a single perturbation generalizes well across unseen data points and different neural network architectures.
  arXiv  Detail & Related papers  (2021-11-19T16:01:45Z)
• With Great Dispersion Comes Greater Resilience: Efficient Poisoning Attacks and Defenses for Linear Regression Models [28.680562906669216]
  We analyze how attackers may interfere with the results of regression learning by poisoning datasets. Our attack, termed Nopt, can produce larger errors with the same proportion of poisoning data-points. Our new defense algorithm, termed Proda, demonstrates an increased effectiveness in reducing errors.
  arXiv  Detail & Related papers  (2020-06-21T22:36:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.