To share or not to share: What risks would laypeople accept to give sensitive data to differentially-private NLP systems?

• URL: http://arxiv.org/abs/2307.06708v2
• Date: Mon, 25 Mar 2024 08:44:53 GMT
• Title: To share or not to share: What risks would laypeople accept to give sensitive data to differentially-private NLP systems?
• Authors: Christopher Weiss, Frauke Kreuter, Ivan Habernal,
• Abstract summary: We argue that determining the $varepsilon$ value should not be solely in the hands of researchers or system developers. We conduct a behavioral experiment (311 lay participants) to study the behavior of people in uncertain decision-making situations.
• Score: 14.586789605230672
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Although the NLP community has adopted central differential privacy as a go-to framework for privacy-preserving model training or data sharing, the choice and interpretation of the key parameter, privacy budget $\varepsilon$ that governs the strength of privacy protection, remains largely arbitrary. We argue that determining the $\varepsilon$ value should not be solely in the hands of researchers or system developers, but must also take into account the actual people who share their potentially sensitive data. In other words: Would you share your instant messages for $\varepsilon$ of 10? We address this research gap by designing, implementing, and conducting a behavioral experiment (311 lay participants) to study the behavior of people in uncertain decision-making situations with respect to privacy-threatening situations. Framing the risk perception in terms of two realistic NLP scenarios and using a vignette behavioral study help us determine what $\varepsilon$ thresholds would lead lay people to be willing to share sensitive textual data - to our knowledge, the first study of its kind.

Related papers

• Calibrating Practical Privacy Risks for Differentially Private Machine Learning [5.363664265121231]
  We study the approaches that can lower the attacking success rate to allow for more flexible privacy budget settings in model training. We find that by selectively suppressing privacy-sensitive features, we can achieve lower ASR values without compromising application-specific data utility.
  arXiv  Detail & Related papers  (2024-10-30T03:52:01Z)
• From Theory to Comprehension: A Comparative Study of Differential Privacy and $k$-Anonymity [2.66269503676104]
  We study users' comprehension of privacy protection provided by a differential privacy mechanism. Our findings suggest that participants' comprehension of differential privacy protection is enhanced by the privacy risk model. Our results confirm our intuition that privacy protection provided by $k$-anonymity is more comprehensible.
  arXiv  Detail & Related papers  (2024-04-05T10:30:26Z)
• What Are the Chances? Explaining the Epsilon Parameter in Differential Privacy [17.201862983773662]
  Differential privacy (DP) is a mathematical privacy notion increasingly deployed across government and industry. We develop three methods to convey probabilistic DP guarantees to end users. We find that odds-based explanation methods are more effective than output-based methods.
  arXiv  Detail & Related papers  (2023-03-01T18:53:25Z)
• Analyzing Privacy Leakage in Machine Learning via Multiple Hypothesis Testing: A Lesson From Fano [83.5933307263932]
  We study data reconstruction attacks for discrete data and analyze it under the framework of hypothesis testing. We show that if the underlying private data takes values from a set of size $M$, then the target privacy parameter $epsilon$ can be $O(log M)$ before the adversary gains significant inferential power.
  arXiv  Detail & Related papers  (2022-10-24T23:50:12Z)
• Smooth Anonymity for Sparse Graphs [69.1048938123063]
  differential privacy has emerged as the gold standard of privacy, however, when it comes to sharing sparse datasets. In this work, we consider a variation of $k$-anonymity, which we call smooth-$k$-anonymity, and design simple large-scale algorithms that efficiently provide smooth-$k$-anonymity.
  arXiv  Detail & Related papers  (2022-07-13T17:09:25Z)
• Individual Privacy Accounting for Differentially Private Stochastic Gradient Descent [69.14164921515949]
  We characterize privacy guarantees for individual examples when releasing models trained by DP-SGD. We find that most examples enjoy stronger privacy guarantees than the worst-case bound. This implies groups that are underserved in terms of model utility simultaneously experience weaker privacy guarantees.
  arXiv  Detail & Related papers  (2022-06-06T13:49:37Z)
• Privacy Amplification via Shuffling for Linear Contextual Bandits [51.94904361874446]
  We study the contextual linear bandit problem with differential privacy (DP) We show that it is possible to achieve a privacy/utility trade-off between JDP and LDP by leveraging the shuffle model of privacy. Our result shows that it is possible to obtain a tradeoff between JDP and LDP by leveraging the shuffle model while preserving local privacy.
  arXiv  Detail & Related papers  (2021-12-11T15:23:28Z)
• Quantifying identifiability to choose and audit $\epsilon$ in differentially private deep learning [15.294433619347082]
  To use differential privacy in machine learning, data scientists must choose privacy parameters $(epsilon,delta)$. We transform $(epsilon,delta)$ to a bound on the Bayesian posterior belief of the adversary assumed by differential privacy concerning the presence of any record in the training dataset. We formulate an implementation of this differential privacy adversary that allows data scientists to audit model training and compute empirical identifiability scores and empirical $(epsilon,delta)$.
  arXiv  Detail & Related papers  (2021-03-04T09:35:58Z)
• Robustness Threats of Differential Privacy [70.818129585404]
  We experimentally demonstrate that networks, trained with differential privacy, in some settings might be even more vulnerable in comparison to non-private versions. We study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect the robustness of the model.
  arXiv  Detail & Related papers  (2020-12-14T18:59:24Z)
• Private Reinforcement Learning with PAC and Regret Guarantees [69.4202374491817]
  We design privacy preserving exploration policies for episodic reinforcement learning (RL) We first provide a meaningful privacy formulation using the notion of joint differential privacy (JDP) We then develop a private optimism-based learning algorithm that simultaneously achieves strong PAC and regret bounds, and enjoys a JDP guarantee.
  arXiv  Detail & Related papers  (2020-09-18T20:18:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.