Tight Auditing of Differentially Private Machine Learning

• URL: http://arxiv.org/abs/2302.07956v1
• Date: Wed, 15 Feb 2023 21:40:33 GMT
• Title: Tight Auditing of Differentially Private Machine Learning
• Authors: Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tram\`er, Matthew Jagielski, Nicholas Carlini, Andreas Terzis
• Abstract summary: For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
• Score: 77.38590306275877
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Auditing mechanisms for differential privacy use probabilistic means to empirically estimate the privacy level of an algorithm. For private machine learning, existing auditing mechanisms are tight: the empirical privacy estimate (nearly) matches the algorithm's provable privacy guarantee. But these auditing techniques suffer from two limitations. First, they only give tight estimates under implausible worst-case assumptions (e.g., a fully adversarial dataset). Second, they require thousands or millions of training runs to produce non-trivial statistical estimates of the privacy leakage. This work addresses both issues. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets -- if the adversary can see all model updates during training. Prior auditing works rely on the same assumption, which is permitted under the standard differential privacy threat model. This threat model is also applicable, e.g., in federated learning settings. Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy. We demonstrate the utility of our improved auditing schemes by surfacing implementation bugs in private machine learning code that eluded prior auditing techniques.

Related papers

• Auditing $f$-Differential Privacy in One Run [43.34594422920125]
  Empirical auditing has emerged as a means of catching some of the flaws in the implementation of privacy-preserving algorithms. We present a tight and efficient auditing procedure and analysis that can effectively assess the privacy of mechanisms.
  arXiv  Detail & Related papers  (2024-10-29T17:02:22Z)
• Auditing Private Prediction [45.23153167486169]
  We study the privacy leakage of four private prediction algorithms: PATE, CaPC, PromptPATE and Private-kNN. Our experiments show that (i) the privacy analysis of private prediction can be improved, (ii) algorithms which are easier to poison lead to much higher privacyleakage, and (iii) the privacy leakage is significantly lower for adversaries without query control than thosewith full control.
  arXiv  Detail & Related papers  (2024-02-14T18:59:27Z)
• Pre-trained Encoders in Self-Supervised Learning Improve Secure and Privacy-preserving Supervised Learning [63.45532264721498]
  Self-supervised learning is an emerging technique to pre-train encoders using unlabeled data. We perform first systematic, principled measurement study to understand whether and when a pretrained encoder can address the limitations of secure or privacy-preserving supervised learning algorithms.
  arXiv  Detail & Related papers  (2022-12-06T21:35:35Z)
• A General Framework for Auditing Differentially Private Machine Learning [27.99806936918949]
  We present a framework to statistically audit the privacy guarantee conferred by a differentially private machine learner in practice. Our work develops a general methodology to empirically evaluate the privacy of differentially private machine learning implementations.
  arXiv  Detail & Related papers  (2022-10-16T21:34:18Z)
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
  We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-06T13:49:37Z)
• Debugging Differential Privacy: A Case Study for Privacy Auditing [60.87570714269048]
  We show that auditing can also be used to find flaws in (purportedly) differentially private schemes. In this case study, we audit a recent open source implementation of a differentially private deep learning algorithm and find, with 99.99999999% confidence, that the implementation does not satisfy the claimed differential privacy guarantee.
  arXiv  Detail & Related papers  (2022-02-24T17:31:08Z)
• Statistical Privacy Guarantees of Machine Learning Preprocessing Techniques [1.198727138090351]
  We adapt a privacy violation detection framework based on statistical methods to measure privacy levels of machine learning pipelines. We apply the newly created framework to show that resampling techniques used when dealing with imbalanced datasets cause the resultant model to leak more privacy.
  arXiv  Detail & Related papers  (2021-09-06T14:08:47Z)
• Private Prediction Sets [72.75711776601973]
  Machine learning systems need reliable uncertainty quantification and protection of individuals' privacy. We present a framework that treats these two desiderata jointly. We evaluate the method on large-scale computer vision datasets.
  arXiv  Detail & Related papers  (2021-02-11T18:59:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.