ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries

• URL: http://arxiv.org/abs/2405.01716v1
• Date: Thu, 2 May 2024 20:23:23 GMT
• Title: ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries
• Authors: Rachel Cummings, Shlomi Hod, Jayshree Sarathy, Marika Swanberg,
• Abstract summary: We offer a detailed taxonomy of attacks, showing the various dimensions of attacks and highlighting that many real-world settings have been understudied. We operationalize our taxonomy by using it to analyze a real-world case study, the Israeli Ministry of Health's recent release of a birth dataset using Differential Privacy.
• Score: 11.550822252074733
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Differential Privacy (DP) is a mathematical framework that is increasingly deployed to mitigate privacy risks associated with machine learning and statistical analyses. Despite the growing adoption of DP, its technical privacy parameters do not lend themselves to an intelligible description of the real-world privacy risks associated with that deployment: the guarantee that most naturally follows from the DP definition is protection against membership inference by an adversary who knows all but one data record and has unlimited auxiliary knowledge. In many settings, this adversary is far too strong to inform how to set real-world privacy parameters. One approach for contextualizing privacy parameters is via defining and measuring the success of technical attacks, but doing so requires a systematic categorization of the relevant attack space. In this work, we offer a detailed taxonomy of attacks, showing the various dimensions of attacks and highlighting that many real-world settings have been understudied. Our taxonomy provides a roadmap for analyzing real-world deployments and developing theoretical bounds for more informative privacy attacks. We operationalize our taxonomy by using it to analyze a real-world case study, the Israeli Ministry of Health's recent release of a birth dataset using DP, showing how the taxonomy enables fine-grained threat modeling and provides insight towards making informed privacy parameter choices. Finally, we leverage the taxonomy towards defining a more realistic attack than previously considered in the literature, namely a distributional reconstruction attack: we generalize Balle et al.'s notion of reconstruction robustness to a less-informed adversary with distributional uncertainty, and extend the worst-case guarantees of DP to this average-case setting.

Related papers

• Privacy-Preserving Dynamic Assortment Selection [4.399892832075127]
  This paper presents a novel framework for privacy-preserving dynamic assortment selection using the multinomial logit (MNL) bandits model. Our approach integrates noise into user utility estimates to balance between exploration and exploitation while ensuring robust privacy protection.
  arXiv  Detail & Related papers  (2024-10-29T19:28:01Z)
• Bayes-Nash Generative Privacy Protection Against Membership Inference Attacks [24.330984323956173]
  We propose a game model for privacy-preserving publishing of data-sharing mechanism outputs. We introduce the notions of Bayes-Nash generative privacy (BNGP) and Bayes generative privacy (BGP) risk. We apply our method to sharing summary statistics, where MIAs can re-identify individuals even from aggregated data.
  arXiv  Detail & Related papers  (2024-10-09T20:29:04Z)
• A Game-Theoretic Approach to Privacy-Utility Tradeoff in Sharing Genomic Summary Statistics [24.330984323956173]
  We propose a game-theoretic framework for optimal privacy-utility tradeoffs in the sharing of genomic summary statistics. Our experiments demonstrate that the proposed framework yields both stronger attacks and stronger defense strategies than the state of the art.
  arXiv  Detail & Related papers  (2024-06-03T22:09:47Z)
• Secure Aggregation is Not Private Against Membership Inference Attacks [66.59892736942953]
  We investigate the privacy implications of SecAgg in federated learning. We show that SecAgg offers weak privacy against membership inference attacks even in a single training round. Our findings underscore the imperative for additional privacy-enhancing mechanisms, such as noise injection.
  arXiv  Detail & Related papers  (2024-03-26T15:07:58Z)
• Reconciling AI Performance and Data Reconstruction Resilience for Medical Imaging [52.578054703818125]
  Artificial Intelligence (AI) models are vulnerable to information leakage of their training data, which can be highly sensitive. Differential Privacy (DP) aims to circumvent these susceptibilities by setting a quantifiable privacy budget. We show that using very large privacy budgets can render reconstruction attacks impossible, while drops in performance are negligible.
  arXiv  Detail & Related papers  (2023-12-05T12:21:30Z)
• Discriminative Adversarial Privacy: Balancing Accuracy and Membership Privacy in Neural Networks [7.0895962209555465]
  Discriminative Adversarial Privacy (DAP) is a learning technique designed to achieve a balance between model performance, speed, and privacy. DAP relies on adversarial training based on a novel loss function able to minimise the prediction error while maximising the MIA's error. In addition, we introduce a novel metric named Accuracy Over Privacy (AOP) to capture the performance-privacy trade-off.
  arXiv  Detail & Related papers  (2023-06-05T17:25:45Z)
• Theoretically Principled Federated Learning for Balancing Privacy and Utility [61.03993520243198]
  We propose a general learning framework for the protection mechanisms that protects privacy via distorting model parameters. It can achieve personalized utility-privacy trade-off for each model parameter, on each client, at each communication round in federated learning.
  arXiv  Detail & Related papers  (2023-05-24T13:44:02Z)
• Is Vertical Logistic Regression Privacy-Preserving? A Comprehensive Privacy Analysis and Beyond [57.10914865054868]
  We consider vertical logistic regression (VLR) trained with mini-batch descent gradient. We provide a comprehensive and rigorous privacy analysis of VLR in a class of open-source Federated Learning frameworks.
  arXiv  Detail & Related papers  (2022-07-19T05:47:30Z)
• Defending against Reconstruction Attacks with R\'enyi Differential Privacy [72.1188520352079]
  Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model. Differential privacy is a known solution to such attacks, but is often used with a relatively large privacy budget. We show that, for a same mechanism, we can derive privacy guarantees for reconstruction attacks that are better than the traditional ones from the literature.
  arXiv  Detail & Related papers  (2022-02-15T18:09:30Z)
• Semantics-Preserved Distortion for Personal Privacy Protection in Information Management [65.08939490413037]
  This paper suggests a linguistically-grounded approach to distort texts while maintaining semantic integrity. We present two distinct frameworks for semantic-preserving distortion: a generative approach and a substitutive approach. We also explore privacy protection in a specific medical information management scenario, showing our method effectively limits sensitive data memorization.
  arXiv  Detail & Related papers  (2022-01-04T04:01:05Z)
• Federated Deep Learning with Bayesian Privacy [28.99404058773532]
  Federated learning (FL) aims to protect data privacy by cooperatively learning a model without sharing private data among users. Homomorphic encryption (HE) based methods provide secure privacy protections but suffer from extremely high computational and communication overheads. Deep learning with Differential Privacy (DP) was implemented as a practical learning algorithm at a manageable cost in complexity.
  arXiv  Detail & Related papers  (2021-09-27T12:48:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.