How reparametrization trick broke differentially-private text representation leaning

• URL: http://arxiv.org/abs/2202.12138v1
• Date: Thu, 24 Feb 2022 15:02:42 GMT
• Title: How reparametrization trick broke differentially-private text representation leaning
• Authors: Ivan Habernal
• Abstract summary: differential privacy is one of the favorite approaches to privacy-preserving methods in NLP. Despite its simplicity, it seems non-trivial to get it right when applying it to NLP. Our main goal is to raise awareness and help the community understand potential pitfalls of applying differential privacy to text representation learning.
• Score: 2.45626162429986
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: As privacy gains traction in the NLP community, researchers have started adopting various approaches to privacy-preserving methods. One of the favorite privacy frameworks, differential privacy (DP), is perhaps the most compelling thanks to its fundamental theoretical guarantees. Despite the apparent simplicity of the general concept of differential privacy, it seems non-trivial to get it right when applying it to NLP. In this short paper, we formally analyze several recent NLP papers proposing text representation learning using DPText (Beigi et al., 2019a,b; Alnasser et al., 2021; Beigi et al., 2021) and reveal their false claims of being differentially private. Furthermore, we also show a simple yet general empirical sanity check to determine whether a given implementation of a DP mechanism almost certainly violates the privacy loss guarantees. Our main goal is to raise awareness and help the community understand potential pitfalls of applying differential privacy to text representation learning.

Related papers

• Differential Privacy Overview and Fundamental Techniques [63.0409690498569]
  This chapter is meant to be part of the book "Differential Privacy in Artificial Intelligence: From Theory to Practice" It starts by illustrating various attempts to protect data privacy, emphasizing where and why they failed. It then defines the key actors, tasks, and scopes that make up the domain of privacy-preserving data analysis.
  arXiv  Detail & Related papers  (2024-11-07T13:52:11Z)
• Thinking Outside of the Differential Privacy Box: A Case Study in Text Privatization with Language Model Prompting [3.3916160303055567]
  We discuss the restrictions that Differential Privacy (DP) integration imposes, as well as bring to light the challenges that such restrictions entail. Our results demonstrate the need for more discussion on the usability of DP in NLP and its benefits over non-DP approaches.
  arXiv  Detail & Related papers  (2024-10-01T14:46:15Z)
• A Statistical Viewpoint on Differential Privacy: Hypothesis Testing, Representation and Blackwell's Theorem [30.365274034429508]
  We argue that differential privacy can be considered a textitpure statistical concept. $f$-differential privacy is a unified framework for analyzing privacy bounds in data analysis and machine learning.
  arXiv  Detail & Related papers  (2024-09-14T23:47:22Z)
• Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory [82.7042006247124]
  We show that even the most capable AI models reveal private information in contexts that humans would not, 39% and 57% of the time, respectively. Our work underscores the immediate need to explore novel inference-time privacy-preserving approaches, based on reasoning and theory of mind.
  arXiv  Detail & Related papers  (2023-10-27T04:15:30Z)
• On Differentially Private Online Predictions [74.01773626153098]
  We introduce an interactive variant of joint differential privacy towards handling online processes. We demonstrate that it satisfies (suitable variants) of group privacy, composition, and post processing. We then study the cost of interactive joint privacy in the basic setting of online classification.
  arXiv  Detail & Related papers  (2023-02-27T19:18:01Z)
• DP-BART for Privatized Text Rewriting under Local Differential Privacy [2.45626162429986]
  We propose a new system 'DP-BART' that largely outperforms existing LDP systems. Our approach uses a novel clipping method, iterative pruning, and further training of internal representations which drastically reduces the amount of noise required for DP guarantees.
  arXiv  Detail & Related papers  (2023-02-15T13:07:34Z)
• Algorithms with More Granular Differential Privacy Guarantees [65.3684804101664]
  We consider partial differential privacy (DP), which allows quantifying the privacy guarantee on a per-attribute basis. In this work, we study several basic data analysis and learning tasks, and design algorithms whose per-attribute privacy parameter is smaller that the best possible privacy parameter for the entire record of a person.
  arXiv  Detail & Related papers  (2022-09-08T22:43:50Z)
• Differential Privacy: What is all the noise about? [0.0]
  Differential Privacy (DP) is a formal definition of privacy that provides rigorous guarantees against risks of privacy breaches during data processing. This paper aims to provide an overview of the most important ideas, concepts and uses of DP in Machine Learning (ML)
  arXiv  Detail & Related papers  (2022-05-19T10:12:29Z)
• Privacy Amplification via Shuffling for Linear Contextual Bandits [51.94904361874446]
  We study the contextual linear bandit problem with differential privacy (DP) We show that it is possible to achieve a privacy/utility trade-off between JDP and LDP by leveraging the shuffle model of privacy. Our result shows that it is possible to obtain a tradeoff between JDP and LDP by leveraging the shuffle model while preserving local privacy.
  arXiv  Detail & Related papers  (2021-12-11T15:23:28Z)
• When differential privacy meets NLP: The devil is in the detail [3.5503507997334958]
  We present a formal analysis of ADePT, a differentially private auto-encoder for text rewriting. Our proof reveals that ADePT is not differentially private, thus rendering the experimental results unsubstantiated.
  arXiv  Detail & Related papers  (2021-09-07T16:12:25Z)
• Private Reinforcement Learning with PAC and Regret Guarantees [69.4202374491817]
  We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
  arXiv  Detail & Related papers  (2020-09-18T20:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.