Revealing and Protecting Labels in Distributed Training
- URL: http://arxiv.org/abs/2111.00556v1
- Date: Sun, 31 Oct 2021 17:57:49 GMT
- Title: Revealing and Protecting Labels in Distributed Training
- Authors: Trung Dang, Om Thakkar, Swaroop Ramaswamy, Rajiv Mathews, Peter Chin,
Fran\c{c}oise Beaufays
- Abstract summary: We propose a method to discover the set of labels of training samples from only the gradient of the last layer and the id to label mapping.
We demonstrate the effectiveness of our method for model training in two domains - image classification, and automatic speech recognition.
- Score: 3.18475216176047
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Distributed learning paradigms such as federated learning often involve
transmission of model updates, or gradients, over a network, thereby avoiding
transmission of private data. However, it is possible for sensitive information
about the training data to be revealed from such gradients. Prior works have
demonstrated that labels can be revealed analytically from the last layer of
certain models (e.g., ResNet), or they can be reconstructed jointly with model
inputs by using Gradients Matching [Zhu et al'19] with additional knowledge
about the current state of the model. In this work, we propose a method to
discover the set of labels of training samples from only the gradient of the
last layer and the id to label mapping. Our method is applicable to a wide
variety of model architectures across multiple domains. We demonstrate the
effectiveness of our method for model training in two domains - image
classification, and automatic speech recognition. Furthermore, we show that
existing reconstruction techniques improve their efficacy when used in
conjunction with our method. Conversely, we demonstrate that gradient
quantization and sparsification can significantly reduce the success of the
attack.
Related papers
- Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning
Interference with Gradient Projection [56.292071534857946]
Recent data-privacy laws have sparked interest in machine unlearning.
Challenge is to discard information about the forget'' data without altering knowledge about remaining dataset.
We adopt a projected-gradient based learning method, named as Projected-Gradient Unlearning (PGU)
We provide empirically evidence to demonstrate that our unlearning method can produce models that behave similar to models retrained from scratch across various metrics even when the training dataset is no longer accessible.
arXiv Detail & Related papers (2023-12-07T07:17:24Z) - Probing the Purview of Neural Networks via Gradient Analysis [13.800680101300756]
We analyze the data-dependent capacity of neural networks and assess anomalies in inputs from the perspective of networks during inference.
To probe the purview of a network, we utilize gradients to measure the amount of change required for the model to characterize the given inputs more accurately.
We demonstrate that our gradient-based approach can effectively differentiate inputs that cannot be accurately represented with learned features.
arXiv Detail & Related papers (2023-04-06T03:02:05Z) - ContraFeat: Contrasting Deep Features for Semantic Discovery [102.4163768995288]
StyleGAN has shown strong potential for disentangled semantic control.
Existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results.
We propose a model that automates this process and achieves state-of-the-art semantic discovery performance.
arXiv Detail & Related papers (2022-12-14T15:22:13Z) - Gradient-Based Adversarial and Out-of-Distribution Detection [15.510581400494207]
We introduce confounding labels in gradient generation to probe the effective expressivity of neural networks.
We show that our gradient-based approach allows for capturing the anomaly in inputs based on the effective expressivity of the models.
arXiv Detail & Related papers (2022-06-16T15:50:41Z) - A self-training framework for glaucoma grading in OCT B-scans [6.382852973055393]
We present a self-training-based framework for glaucoma grading using OCT B-scans under the presence of domain shift.
A two-step learning methodology resorts to pseudo-labels generated during the first step to augment the training dataset on the target domain.
We propose a novel glaucoma-specific backbone which introduces residual and attention modules via skip-connections to refine the embedding features of the latent space.
arXiv Detail & Related papers (2021-11-23T10:33:55Z) - Understanding Training-Data Leakage from Gradients in Neural Networks
for Image Classification [11.272188531829016]
In many applications, we need to protect the training data from being leaked due to IP or privacy concerns.
Recent works have demonstrated that it is possible to reconstruct the training data from gradients for an image-classification model when its architecture is known.
We formulate the problem of training data reconstruction as solving an optimisation problem iteratively for each layer.
We are able to attribute the potential leakage of the training data in a deep network to its architecture.
arXiv Detail & Related papers (2021-11-19T12:14:43Z) - Style Curriculum Learning for Robust Medical Image Segmentation [62.02435329931057]
Deep segmentation models often degrade due to distribution shifts in image intensities between the training and test data sets.
We propose a novel framework to ensure robust segmentation in the presence of such distribution shifts.
arXiv Detail & Related papers (2021-08-01T08:56:24Z) - Semi-Supervised Domain Adaptation with Prototypical Alignment and
Consistency Learning [86.6929930921905]
This paper studies how much it can help address domain shifts if we further have a few target samples labeled.
To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks.
Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
arXiv Detail & Related papers (2021-04-19T08:46:08Z) - Learning Propagation Rules for Attribution Map Generation [146.71503336770886]
We propose a dedicated method to generate attribution maps that allow us to learn the propagation rules automatically.
Specifically, we introduce a learnable plugin module, which enables adaptive propagation rules for each pixel.
The introduced learnable module can be trained under any auto-grad framework with higher-order differential support.
arXiv Detail & Related papers (2020-10-14T16:23:58Z) - Region Comparison Network for Interpretable Few-shot Image
Classification [97.97902360117368]
Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes.
We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works.
We also present a new way to generalize the interpretability from the level of tasks to categories.
arXiv Detail & Related papers (2020-09-08T07:29:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.