Debugging Differential Privacy: A Case Study for Privacy Auditing

• URL: http://arxiv.org/abs/2202.12219v1
• Date: Thu, 24 Feb 2022 17:31:08 GMT
• Title: Debugging Differential Privacy: A Case Study for Privacy Auditing
• Authors: Florian Tramer, Andreas Terzis, Thomas Steinke, Shuang Song, Matthew Jagielski, Nicholas Carlini
• Abstract summary: We show that auditing can also be used to find flaws in (purportedly) differentially private schemes. In this case study, we audit a recent open source implementation of a differentially private deep learning algorithm and find, with 99.99999999% confidence, that the implementation does not satisfy the claimed differential privacy guarantee.
• Score: 60.87570714269048
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differential Privacy can provide provable privacy guarantees for training data in machine learning. However, the presence of proofs does not preclude the presence of errors. Inspired by recent advances in auditing which have been used for estimating lower bounds on differentially private algorithms, here we show that auditing can also be used to find flaws in (purportedly) differentially private schemes. In this case study, we audit a recent open source implementation of a differentially private deep learning algorithm and find, with 99.99999999% confidence, that the implementation does not satisfy the claimed differential privacy guarantee.

Related papers

• Privacy Audit as Bits Transmission: (Im)possibilities for Audit by One Run [7.850976675388593]
  We introduce a unifying framework for privacy audits based on information-theoretic principles. We demystify the method of privacy audit by one run, identifying the conditions under which single-run audits are feasible or infeasible.
  arXiv  Detail & Related papers  (2025-01-29T16:38:51Z)
• Differentially Private Random Feature Model [52.468511541184895]
  We produce a differentially private random feature model for privacy-preserving kernel machines. We show that our method preserves privacy and derive a generalization error bound for the method.
  arXiv  Detail & Related papers  (2024-12-06T05:31:08Z)
• Uncertainty quantification by block bootstrap for differentially private stochastic gradient descent [1.0742675209112622]
  Gradient Descent (SGD) is a widely used tool in machine learning. Uncertainty quantification (UQ) for SGD by bootstrap has been addressed by several authors. We propose a novel block bootstrap for SGD under local differential privacy.
  arXiv  Detail & Related papers  (2024-05-21T07:47:21Z)
• Training Private Models That Know What They Don't Know [40.19666295972155]
  We find that several popular selective prediction approaches are ineffective in a differentially private setting. We propose a novel evaluation mechanism which isolate selective prediction performance across model utility levels.
  arXiv  Detail & Related papers  (2023-05-28T12:20:07Z)
• Tight Auditing of Differentially Private Machine Learning [77.38590306275877]
  For private machine learning, existing auditing mechanisms are tight. They only give tight estimates under implausible worst-case assumptions. We design an improved auditing scheme that yields tight privacy estimates for natural (not adversarially crafted) datasets.
  arXiv  Detail & Related papers  (2023-02-15T21:40:33Z)
• A General Framework for Auditing Differentially Private Machine Learning [27.99806936918949]
  We present a framework to statistically audit the privacy guarantee conferred by a differentially private machine learner in practice. Our work develops a general methodology to empirically evaluate the privacy of differentially private machine learning implementations.
  arXiv  Detail & Related papers  (2022-10-16T21:34:18Z)
• Algorithms with More Granular Differential Privacy Guarantees [65.3684804101664]
  We consider partial differential privacy (DP), which allows quantifying the privacy guarantee on a per-attribute basis. In this work, we study several basic data analysis and learning tasks, and design algorithms whose per-attribute privacy parameter is smaller that the best possible privacy parameter for the entire record of a person.
  arXiv  Detail & Related papers  (2022-09-08T22:43:50Z)
• Private Domain Adaptation from a Public Source [48.83724068578305]
  We design differentially private discrepancy-based algorithms for adaptation from a source domain with public labeled data to a target domain with unlabeled private data. Our solutions are based on private variants of Frank-Wolfe and Mirror-Descent algorithms.
  arXiv  Detail & Related papers  (2022-08-12T06:52:55Z)
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
  We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-06T13:49:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.