Enforcing Demographic Coherence: A Harms Aware Framework for Reasoning about Private Data Release

• URL: http://arxiv.org/abs/2502.02709v1
• Date: Tue, 04 Feb 2025 20:42:30 GMT
• Title: Enforcing Demographic Coherence: A Harms Aware Framework for Reasoning about Private Data Release
• Authors: Mark Bun, Marco Carmosino, Palak Jain, Gabriel Kaptchuk, Satchit Sivakumar,
• Abstract summary: We introduce demographic coherence, a condition inspired by privacy attacks that we argue is necessary for data privacy. Our framework focuses on confidence rated predictors, which can in turn be distilled from almost any data-informed process. We prove that every differentially private data release is also demographically coherent, and that there are demographically coherent algorithms which are not differentially private.
• Score: 14.939460540040459
• License:
• Abstract: The technical literature about data privacy largely consists of two complementary approaches: formal definitions of conditions sufficient for privacy preservation and attacks that demonstrate privacy breaches. Differential privacy is an accepted standard in the former sphere. However, differential privacy's powerful adversarial model and worst-case guarantees may make it too stringent in some situations, especially when achieving it comes at a significant cost to data utility. Meanwhile, privacy attacks aim to expose real and worrying privacy risks associated with existing data release processes but often face criticism for being unrealistic. Moreover, the literature on attacks generally does not identify what properties are necessary to defend against them. We address the gap between these approaches by introducing demographic coherence, a condition inspired by privacy attacks that we argue is necessary for data privacy. This condition captures privacy violations arising from inferences about individuals that are incoherent with respect to the demographic patterns in the data. Our framework focuses on confidence rated predictors, which can in turn be distilled from almost any data-informed process. Thus, we capture privacy threats that exist even when no attack is explicitly being carried out. Our framework not only provides a condition with respect to which data release algorithms can be analysed but suggests natural experimental evaluation methodologies that could be used to build practical intuition and make tangible assessment of risks. Finally, we argue that demographic coherence is weaker than differential privacy: we prove that every differentially private data release is also demographically coherent, and that there are demographically coherent algorithms which are not differentially private.

Related papers

• Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy [64.32494202656801]
  Privacy-preserving computer vision is an important emerging problem in machine learning and artificial intelligence. We present anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context. We also proposeMaskDP to protect non-anonymized but privacy sensitive background information.
  arXiv  Detail & Related papers  (2024-10-22T15:22:53Z)
• RASE: Efficient Privacy-preserving Data Aggregation against Disclosure Attacks for IoTs [2.1765174838950494]
  We study the new paradigm for collecting and protecting the data produced by ever-increasing sensor devices. Most previous studies on co-design of data aggregation and privacy preservation assume that a trusted fusion center adheres to privacy regimes. We propose a novel paradigm (called RASE), which can be generalized into a 3-step sequential procedure, noise addition, followed by random permutation, and then parameter estimation.
  arXiv  Detail & Related papers  (2024-05-31T15:21:38Z)
• A Summary of Privacy-Preserving Data Publishing in the Local Setting [0.6749750044497732]
  Statistical Disclosure Control aims to minimize the risk of exposing confidential information by de-identifying it. We outline the current privacy-preserving techniques employed in microdata de-identification, delve into privacy measures tailored for various disclosure scenarios, and assess metrics for information loss and predictive performance.
  arXiv  Detail & Related papers  (2023-12-19T04:23:23Z)
• $\alpha$-Mutual Information: A Tunable Privacy Measure for Privacy Protection in Data Sharing [4.475091558538915]
  This paper adopts Arimoto's $alpha$-Mutual Information as a tunable privacy measure. We formulate a general distortion-based mechanism that manipulates the original data to offer privacy protection.
  arXiv  Detail & Related papers  (2023-10-27T16:26:14Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• A Unified View of Differentially Private Deep Generative Modeling [60.72161965018005]
  Data with privacy concerns comes with stringent regulations that frequently prohibited data access and data sharing. Overcoming these obstacles is key for technological progress in many real-world application scenarios that involve privacy sensitive data. Differentially private (DP) data publishing provides a compelling solution, where only a sanitized form of the data is publicly released.
  arXiv  Detail & Related papers  (2023-09-27T14:38:16Z)
• Enabling Trade-offs in Privacy and Utility in Genomic Data Beacons and Summary Statistics [26.99521354120141]
  We introduce optimization-based approaches to explicitly trade off the utility of summary data or Beacon responses and privacy. In the first, an attacker applies a likelihood-ratio test to make membership-inference claims. In the second, an attacker uses a threshold that accounts for the effect of the data release on the separation in scores between individuals.
  arXiv  Detail & Related papers  (2023-01-11T19:16:13Z)
• How Do Input Attributes Impact the Privacy Loss in Differential Privacy? [55.492422758737575]
  We study the connection between the per-subject norm in DP neural networks and individual privacy loss. We introduce a novel metric termed the Privacy Loss-Input Susceptibility (PLIS) which allows one to apportion the subject's privacy loss to their input attributes.
  arXiv  Detail & Related papers  (2022-11-18T11:39:03Z)
• No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy" [75.98836424725437]
  New methods designed to preserve data privacy require careful scrutiny. Failure to preserve privacy is hard to detect, and yet can lead to catastrophic results when a system implementing a privacy-preserving'' method is attacked.
  arXiv  Detail & Related papers  (2022-09-29T17:50:23Z)
• Robustness Threats of Differential Privacy [70.818129585404]
  We experimentally demonstrate that networks, trained with differential privacy, in some settings might be even more vulnerable in comparison to non-private versions. We study how the main ingredients of differentially private neural networks training, such as gradient clipping and noise addition, affect the robustness of the model.
  arXiv  Detail & Related papers  (2020-12-14T18:59:24Z)
• Data-driven Regularized Inference Privacy [33.71757542373714]
  We propose a data-driven inference privacy preserving framework to sanitize data. We develop an inference privacy framework based on the variational method. We present empirical methods to estimate the privacy metric.
  arXiv  Detail & Related papers  (2020-10-10T08:42:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.