Auditing Differentially Private Machine Learning: How Private is Private SGD?

• URL: http://arxiv.org/abs/2006.07709v1
• Date: Sat, 13 Jun 2020 20:00:18 GMT
• Title: Auditing Differentially Private Machine Learning: How Private is Private SGD?
• Authors: Matthew Jagielski and Jonathan Ullman and Alina Oprea
• Abstract summary: We investigate whether Differentially Private SGD offers better privacy in practice than what is guaranteed by its state-of-the-art analysis. We do so via novel data poisoning attacks, which we show correspond to realistic privacy attacks.
• Score: 16.812900569416062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We investigate whether Differentially Private SGD offers better privacy in practice than what is guaranteed by its state-of-the-art analysis. We do so via novel data poisoning attacks, which we show correspond to realistic privacy attacks. While previous work (Ma et al., arXiv 2019) proposed this connection between differential privacy and data poisoning as a defense against data poisoning, our use as a tool for understanding the privacy of a specific mechanism is new. More generally, our work takes a quantitative, empirical approach to understanding the privacy afforded by specific implementations of differentially private algorithms that we believe has the potential to complement and influence analytical work on differential privacy.

Related papers

• Differential Privacy Overview and Fundamental Techniques [63.0409690498569]
  This chapter is meant to be part of the book "Differential Privacy in Artificial Intelligence: From Theory to Practice" It starts by illustrating various attempts to protect data privacy, emphasizing where and why they failed. It then defines the key actors, tasks, and scopes that make up the domain of privacy-preserving data analysis.
  arXiv  Detail & Related papers  (2024-11-07T13:52:11Z)
• Masked Differential Privacy [64.32494202656801]
  We propose an effective approach called masked differential privacy (DP), which allows for controlling sensitive regions where differential privacy is applied. Our method operates selectively on data and allows for defining non-sensitive-temporal regions without DP application or combining differential privacy with other privacy techniques within data samples.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation and Blackwell's Theorem [30.365274034429508]
  We argue that differential privacy can be considered a textitpure statistical concept. $f$-differential privacy is a unified framework for analyzing privacy bounds in data analysis and machine learning.
  arXiv  Detail & Related papers  (2024-09-14T23:47:22Z)
• Confounding Privacy and Inverse Composition [32.85314813605347]
  In differential privacy, sensitive information is contained in the dataset while in Pufferfish privacy, sensitive information determines data distribution. We introduce a novel privacy notion of ($epsilon, delta$)-confounding privacy that generalizes both differential privacy and Pufferfish privacy.
  arXiv  Detail & Related papers  (2024-08-21T21:45:13Z)
• Models Matter: Setting Accurate Privacy Expectations for Local and Central Differential Privacy [14.40391109414476]
  We design and evaluate new explanations of differential privacy for the local and central models. We find that consequences-focused explanations in the style of privacy nutrition labels are a promising approach for setting accurate privacy expectations.
  arXiv  Detail & Related papers  (2024-08-16T01:21:57Z)
• Adaptive Privacy Composition for Accuracy-first Mechanisms [55.53725113597539]
  Noise reduction mechanisms produce increasingly accurate answers. Analysts only pay the privacy cost of the least noisy or most accurate answer released. There has yet to be any study on how ex-post private mechanisms compose. We develop privacy filters that allow an analyst to adaptively switch between differentially private and ex-post private mechanisms.
  arXiv  Detail & Related papers  (2023-06-24T00:33:34Z)
• Algorithms with More Granular Differential Privacy Guarantees [65.3684804101664]
  We consider partial differential privacy (DP), which allows quantifying the privacy guarantee on a per-attribute basis. In this work, we study several basic data analysis and learning tasks, and design algorithms whose per-attribute privacy parameter is smaller that the best possible privacy parameter for the entire record of a person.
  arXiv  Detail & Related papers  (2022-09-08T22:43:50Z)
• Smooth Anonymity for Sparse Graphs [69.1048938123063]
  differential privacy has emerged as the gold standard of privacy, however, when it comes to sharing sparse datasets. In this work, we consider a variation of $k$-anonymity, which we call smooth-$k$-anonymity, and design simple large-scale algorithms that efficiently provide smooth-$k$-anonymity.
  arXiv  Detail & Related papers  (2022-07-13T17:09:25Z)
• Assessing Differentially Private Variational Autoencoders under Membership Inference [26.480694390462617]
  We quantify and compare the privacy-accuracy trade-off for differentially private Variational Autoencoders. We do rarely observe favorable privacy-accuracy trade-off for Variational Autoencoders, and identify a case where LDP outperforms CDP.
  arXiv  Detail & Related papers  (2022-04-16T21:53:09Z)
• "I need a better description'': An Investigation Into User Expectations For Differential Privacy [31.352325485393074]
  We explore users' privacy expectations related to differential privacy. We find that users care about the kinds of information leaks against which differential privacy protects. We find that the ways in which differential privacy is described in-the-wild haphazardly set users' privacy expectations.
  arXiv  Detail & Related papers  (2021-10-13T02:36:37Z)
• Private Reinforcement Learning with PAC and Regret Guarantees [69.4202374491817]
  We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
  arXiv  Detail & Related papers  (2020-09-18T20:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.