iDLG: Improved Deep Leakage from Gradients
- URL: http://arxiv.org/abs/2001.02610v1
- Date: Wed, 8 Jan 2020 16:45:09 GMT
- Title: iDLG: Improved Deep Leakage from Gradients
- Authors: Bo Zhao, Konda Reddy Mopuri, Hakan Bilen
- Abstract summary: It is widely believed that sharing gradients will not leak private training data in distributed learning systems.
We propose a simple but reliable approach to extract accurate data from the gradients.
Our approach is valid for any differentiable model trained with cross-entropy loss over one-hot labels.
- Score: 36.14340188365505
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: It is widely believed that sharing gradients will not leak private training
data in distributed learning systems such as Collaborative Learning and
Federated Learning, etc. Recently, Zhu et al. presented an approach which shows
the possibility to obtain private training data from the publicly shared
gradients. In their Deep Leakage from Gradient (DLG) method, they synthesize
the dummy data and corresponding labels with the supervision of shared
gradients. However, DLG has difficulty in convergence and discovering the
ground-truth labels consistently. In this paper, we find that sharing gradients
definitely leaks the ground-truth labels. We propose a simple but reliable
approach to extract accurate data from the gradients. Particularly, our
approach can certainly extract the ground-truth labels as opposed to DLG, hence
we name it Improved DLG (iDLG). Our approach is valid for any differentiable
model trained with cross-entropy loss over one-hot labels. We mathematically
illustrate how our method can extract ground-truth labels from the gradients
and empirically demonstrate the advantages over DLG.
Related papers
- Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation [10.404379188947383]
Two-party split learning has been proven to survive label inference attacks.
We propose a novel two-party split learning method to defend against existing label inference attacks.
arXiv Detail & Related papers (2024-10-11T09:25:21Z) - Two Trades is not Baffled: Condensing Graph via Crafting Rational Gradient Matching [50.30124426442228]
Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have raised growing concerns.
We propose a novel graph method named textbfCraftextbfTing textbfRationatextbf (textbfCTRL) which offers an optimized starting point closer to the original dataset's feature distribution.
arXiv Detail & Related papers (2024-02-07T14:49:10Z) - The Manifold Hypothesis for Gradient-Based Explanations [55.01671263121624]
gradient-based explanation algorithms provide perceptually-aligned explanations.
We show that the more a feature attribution is aligned with the tangent space of the data, the more perceptually-aligned it tends to be.
We suggest that explanation algorithms should actively strive to align their explanations with the data manifold.
arXiv Detail & Related papers (2022-06-15T08:49:24Z) - Gradient Inversion Attack: Leaking Private Labels in Two-Party Split
Learning [12.335698325757491]
We propose a label leakage attack that allows an adversarial input owner to learn the label owner's private labels.
Our attack can uncover the private label data on several multi-class image classification problems and a binary conversion prediction task with near-perfect accuracy.
While this technique is effective for simpler datasets, it significantly degrades utility for datasets with higher input dimensionality.
arXiv Detail & Related papers (2021-11-25T16:09:59Z) - Revealing and Protecting Labels in Distributed Training [3.18475216176047]
We propose a method to discover the set of labels of training samples from only the gradient of the last layer and the id to label mapping.
We demonstrate the effectiveness of our method for model training in two domains - image classification, and automatic speech recognition.
arXiv Detail & Related papers (2021-10-31T17:57:49Z) - CAFE: Catastrophic Data Leakage in Vertical Federated Learning [65.56360219908142]
Recent studies show that private training data can be leaked through the gradients sharing mechanism deployed in distributed machine learning systems.
We propose an advanced data leakage attack with theoretical justification to efficiently recover batch data from the shared aggregated gradients.
arXiv Detail & Related papers (2021-10-26T23:22:58Z) - Gradient Imitation Reinforcement Learning for Low Resource Relation
Extraction [52.63803634033647]
Low-resource relation Extraction (LRE) aims to extract relation facts from limited labeled corpora when human annotation is scarce.
We develop a Gradient Imitation Reinforcement Learning method to encourage pseudo label data to imitate the gradient descent direction on labeled data.
We also propose a framework called GradLRE, which handles two major scenarios in low-resource relation extraction.
arXiv Detail & Related papers (2021-09-14T03:51:15Z) - Quantifying Information Leakage from Gradients [8.175697239083474]
Sharing deep neural networks' gradients instead of training data could facilitate data privacy in collaborative learning.
In practice however, gradients can disclose both private latent attributes and original data.
Mathematical metrics are needed to quantify both original and latent information leakages from gradients computed over the training data.
arXiv Detail & Related papers (2021-05-28T15:47:44Z) - User Label Leakage from Gradients in Federated Learning [12.239472997714804]
Federated learning enables multiple users to build a joint model by sharing their model updates (gradients)
We propose Label Leakage from Gradients (LLG), a novel attack to extract the labels of the users' training data from their shared gradients.
arXiv Detail & Related papers (2021-05-19T19:21:05Z) - Understanding Gradient Clipping in Private SGD: A Geometric Perspective [68.61254575987013]
Deep learning models are increasingly popular in many machine learning applications where the training data may contain sensitive information.
Many learning systems now incorporate differential privacy by training their models with (differentially) private SGD.
A key step in each private SGD update is gradient clipping that shrinks the gradient of an individual example whenever its L2 norm exceeds some threshold.
arXiv Detail & Related papers (2020-06-27T19:08:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.