Is Private Learning Possible with Instance Encoding?

• URL: http://arxiv.org/abs/2011.05315v2
• Date: Wed, 28 Apr 2021 01:18:36 GMT
• Title: Is Private Learning Possible with Instance Encoding?
• Authors: Nicholas Carlini, Samuel Deng, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, Shuang Song, Abhradeep Thakurta, Florian Tramer
• Abstract summary: We study whether a non-private learning algorithm can be made private by relying on an instance-encoding mechanism. We formalize both the notion of instance encoding and its privacy by providing two attack models.
• Score: 68.84324434746765
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A private machine learning algorithm hides as much as possible about its training data while still preserving accuracy. In this work, we study whether a non-private learning algorithm can be made private by relying on an instance-encoding mechanism that modifies the training inputs before feeding them to a normal learner. We formalize both the notion of instance encoding and its privacy by providing two attack models. We first prove impossibility results for achieving a (stronger) model. Next, we demonstrate practical attacks in the second (weaker) attack model on InstaHide, a recent proposal by Huang, Song, Li and Arora [ICML'20] that aims to use instance encoding for privacy.

Related papers

• Why Is Public Pretraining Necessary for Private Model Training? [50.054565310457306]
  We show that pretraining on publicly available data leads to distinct gains over nonprivate settings. We argue that the tradeoff may be a deeper loss model that requires an algorithm to go through two phases. Guided by intuition, we provide theoretical constructions that provably demonstrate the separation between private with and without public pretraining.
  arXiv  Detail & Related papers  (2023-02-19T05:32:20Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers [71.70205894168039]
  We consider instance-wise unlearning, of which the goal is to delete information on a set of instances from a pre-trained model. We propose two methods that reduce forgetting on the remaining data: 1) utilizing adversarial examples to overcome forgetting at the representation-level and 2) leveraging weight importance metrics to pinpoint network parameters guilty of propagating unwanted information.
  arXiv  Detail & Related papers  (2023-01-27T07:53:50Z)
• Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning [63.45532264721498]
  Self-supervised learning is an emerging technique to pre-train encoders using unlabeled data. We perform first systematic, principled measurement study to understand whether and when a pretrained encoder can address the limitations of secure or privacy-preserving supervised learning algorithms.
  arXiv  Detail & Related papers  (2022-12-06T21:35:35Z)
• Fine-Tuning with Differential Privacy Necessitates an Additional Hyperparameter Search [38.83524780461911]
  We show how carefully selecting the layers being fine-tuned in the pretrained neural network allows us to establish new state-of-the-art tradeoffs between privacy and accuracy. We achieve 77.9% accuracy for $(varepsilon, delta)= (2, 10-5)$ on CIFAR-100 for a model pretrained on ImageNet.
  arXiv  Detail & Related papers  (2022-10-05T11:32:49Z)
• Defending against Reconstruction Attacks with R\'enyi Differential Privacy [72.1188520352079]
  Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget. We show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature.
  arXiv  Detail & Related papers  (2022-02-15T18:09:30Z)
• On the Importance of Encrypting Deep Features [15.340540198612823]
  We analyze model inversion attacks with only two assumptions: feature vectors of user data are known, and a black-box API for inference is provided. Experiments have been conducted on state-of-the-art models in person re-identification, and two attack scenarios (i.e., recognizing auxiliary attributes and reconstructing user data) are investigated. Results show that an adversary could successfully infer sensitive information even under severe constraints.
  arXiv  Detail & Related papers  (2021-08-16T15:22:33Z)
• Manipulating SGD with Data Ordering Attacks [23.639512087220137]
  We present a class of training-time attacks that require no changes to the underlying model dataset or architecture. In particular, an attacker can disrupt the integrity and availability of a model by simply reordering training batches. Attacks have a long-term impact in that they decrease model performance hundreds of epochs after the attack took place.
  arXiv  Detail & Related papers  (2021-04-19T22:17:27Z)
• When Machine Unlearning Jeopardizes Privacy [25.167214892258567]
  We investigate the unintended information leakage caused by machine unlearning. We propose a novel membership inference attack that achieves strong performance. Our results can help improve privacy protection in practical implementations of machine unlearning.
  arXiv  Detail & Related papers  (2020-05-05T14:11:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.