Students Parrot Their Teachers: Membership Inference on Model Distillation

• URL: http://arxiv.org/abs/2303.03446v1
• Date: Mon, 6 Mar 2023 19:16:23 GMT
• Title: Students Parrot Their Teachers: Membership Inference on Model Distillation
• Authors: Matthew Jagielski, Milad Nasr, Christopher Choquette-Choo, Katherine Lee, Nicholas Carlini
• Abstract summary: We study the privacy provided by knowledge distillation to both the teacher and student training sets. Our attacks are strongest when student and teacher sets are similar, or when the attacker can poison the teacher set.
• Score: 54.392069096234074
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Model distillation is frequently proposed as a technique to reduce the privacy leakage of machine learning. These empirical privacy defenses rely on the intuition that distilled ``student'' models protect the privacy of training data, as they only interact with this data indirectly through a ``teacher'' model. In this work, we design membership inference attacks to systematically study the privacy provided by knowledge distillation to both the teacher and student training sets. Our new attacks show that distillation alone provides only limited privacy across a number of domains. We explain the success of our attacks on distillation by showing that membership inference attacks on a private dataset can succeed even if the target model is *never* queried on any actual training points, but only on inputs whose predictions are highly influenced by training data. Finally, we show that our attacks are strongest when student and teacher sets are similar, or when the attacker can poison the teacher set.

Related papers

• No Vandalism: Privacy-Preserving and Byzantine-Robust Federated Learning [18.1129191782913]
  Federated learning allows several clients to train one machine learning model jointly without sharing private data, providing privacy protection. Traditional federated learning is vulnerable to poisoning attacks, which can not only decrease the model performance, but also implant malicious backdoors. In this paper, we aim to build a privacy-preserving and Byzantine-robust federated learning scheme to provide an environment with no vandalism (NoV) against attacks from malicious participants.
  arXiv  Detail & Related papers  (2024-06-03T07:59:10Z)
• Differentially Private and Adversarially Robust Machine Learning: An Empirical Evaluation [2.8084422332394428]
  Malicious adversaries can attack machine learning models to infer sensitive information or damage the system by launching a series of evasion attacks. This study explores the combination of adversarial training and differentially private training to defend against simultaneous attacks.
  arXiv  Detail & Related papers  (2024-01-18T22:26:31Z)
• Protecting Split Learning by Potential Energy Loss [70.81375125791979]
  We focus on the privacy leakage from the forward embeddings of split learning. We propose the potential energy loss to make the forward embeddings become more 'complicated'
  arXiv  Detail & Related papers  (2022-10-18T06:21:11Z)
• The Privacy Onion Effect: Memorization is Relative [76.46529413546725]
  We show an Onion Effect of memorization: removing the "layer" of outlier points that are most vulnerable exposes a new layer of previously-safe points to the same attack. It suggests that privacy-enhancing technologies such as machine unlearning could actually harm the privacy of other users.
  arXiv  Detail & Related papers  (2022-06-21T15:25:56Z)
• Defense Against Gradient Leakage Attacks via Learning to Obscure Data [48.67836599050032]
  Federated learning is considered as an effective privacy-preserving learning mechanism. In this paper, we propose a new defense method to protect the privacy of clients' data by learning to obscure data.
  arXiv  Detail & Related papers  (2022-06-01T21:03:28Z)
• Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets [53.866927712193416]
  We show that an adversary who can poison a training dataset can cause models trained on this dataset to leak private details belonging to other parties. Our attacks are effective across membership inference, attribute inference, and data extraction. Our results cast doubts on the relevance of cryptographic privacy guarantees in multiparty protocols for machine learning.
  arXiv  Detail & Related papers  (2022-03-31T18:06:28Z)
• FaceLeaks: Inference Attacks against Transfer Learning Models via Black-box Queries [2.7564955518050693]
  We investigate if one can leak or infer private information without interacting with the teacher model directly. We propose novel strategies to infer from aggregate-level information. Our study indicates that information leakage is a real privacy threat to the transfer learning framework widely used in real-life situations.
  arXiv  Detail & Related papers  (2020-10-27T03:02:40Z)
• Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries [74.59376038272661]
  We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model. We show that a victim model that only publishes the labels is still susceptible to sampling attacks and the adversary can recover up to 100% of its performance. For defense, we choose differential privacy in the form of gradient perturbation during the training of the victim model as well as output perturbation at prediction time.
  arXiv  Detail & Related papers  (2020-09-01T12:54:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.