GPM: The Gaussian Pancake Mechanism for Planting Undetectable Backdoors in Differential Privacy

• URL: http://arxiv.org/abs/2509.23834v1
• Date: Sun, 28 Sep 2025 12:14:06 GMT
• Title: GPM: The Gaussian Pancake Mechanism for Planting Undetectable Backdoors in Differential Privacy
• Authors: Haochen Sun, Xi He,
• Abstract summary: Several incidents of unintended privacy loss have occurred due to numerical issues and inappropriate configurations of Differential Privacy software.<n>We present the Gaussian pancake mechanism (GPM), a novel mechanism that is computationally indistinguishable from the widely used Gaussian mechanism (GM)<n>Unlike the unintentional privacy loss caused by GM's numerical issues, GPM is an adversarial yet undetectable backdoor attack against data privacy.
• Score: 4.281902449361707
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Differential privacy (DP) has become the gold standard for preserving individual privacy in data analysis. However, an implicit yet fundamental assumption underlying these rigorous privacy guarantees is the correct implementation and execution of DP mechanisms. Several incidents of unintended privacy loss have occurred due to numerical issues and inappropriate configurations of DP software, which have been successfully exploited in privacy attacks. To better understand the seriousness of defective DP software, we ask the following question: is it possible to elevate these passive defects into active privacy attacks while maintaining covertness? To address this question, we present the Gaussian pancake mechanism (GPM), a novel mechanism that is computationally indistinguishable from the widely used Gaussian mechanism (GM), yet exhibits arbitrarily weaker statistical DP guarantees. This unprecedented separation enables a new class of backdoor attacks: by indistinguishably passing off as the authentic GM, GPM can covertly degrade statistical privacy. Unlike the unintentional privacy loss caused by GM's numerical issues, GPM is an adversarial yet undetectable backdoor attack against data privacy. We formally prove GPM's covertness, characterize its statistical leakage, and demonstrate a concrete distinguishing attack that can achieve near-perfect success rates under suitable parameter choices, both theoretically and empirically. Our results underscore the importance of using transparent, open-source DP libraries and highlight the need for rigorous scrutiny and formal verification of DP implementations to prevent subtle, undetectable privacy compromises in real-world systems.

Related papers

• Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacy [50.66105844449181]
  Individual Differential Privacy (iDP) promises users control over their privacy, but this promise can be broken in practice.<n>We reveal a previously overlooked vulnerability in sampling-based iDP mechanisms.<n>We propose $(varepsilon_i,_i,overline)$-iDP a privacy contract that uses $$-divergences to provide users with a hard upper bound on their excess vulnerability.
  arXiv  Detail & Related papers  (2026-01-19T10:26:12Z)
• Beyond the Worst Case: Extending Differential Privacy Guarantees to Realistic Adversaries [17.780319275883127]
  Differential Privacy is a family of definitions that bound the worst-case privacy leakage of a mechanism.<n>This work sheds light on what the worst-case guarantee of DP implies about the success of attackers that are more representative of real-world privacy risks.
  arXiv  Detail & Related papers  (2025-07-10T20:36:31Z)
• Machine Learning with Privacy for Protected Attributes [56.44253915927481]
  We refine the definition of differential privacy (DP) to create a more general and flexible framework that we call feature differential privacy (FDP)<n>Our definition is simulation-based and allows for both addition/removal and replacement variants of privacy, and can handle arbitrary separation of protected and non-protected features.<n>We apply our framework to various machine learning tasks and show that it can significantly improve the utility of DP-trained models when public features are available.
  arXiv  Detail & Related papers  (2025-06-24T17:53:28Z)
• Gaussian DP for Reporting Differential Privacy Guarantees in Machine Learning [27.52540933835594]
  Current practices for reporting the level of differential privacy for machine learning (ML) algorithms provide an incomplete and potentially misleading picture of the privacy guarantees.<n>We argue that using non-asymptotic Gaussian Differential Privacy (GDP) as the primary means of communicating DP guarantees in ML avoids these potential downsides.<n>We show how to provide non-asymptotic bounds on GDP using numerical accountants, and show that GDP can capture the entire privacy profile of DP-SGD and related algorithms with virtually no error, as quantified by the metric.
  arXiv  Detail & Related papers  (2025-03-13T23:06:30Z)
• It's Our Loss: No Privacy Amplification for Hidden State DP-SGD With Non-Convex Loss [0.76146285961466]
  We show that for specific loss functions, the final iterate of DP-SGD leaks as much information as the final loss function. We conclude that no privacy amplification is possible for DP-SGD in general for all (non-) loss functions.
  arXiv  Detail & Related papers  (2024-07-09T01:58:19Z)
• Privacy Amplification for the Gaussian Mechanism via Bounded Support [64.86780616066575]
  Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. We propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting.
  arXiv  Detail & Related papers  (2024-03-07T21:22:07Z)
• Unified Mechanism-Specific Amplification by Subsampling and Group Privacy Amplification [54.1447806347273]
  Amplification by subsampling is one of the main primitives in machine learning with differential privacy. We propose the first general framework for deriving mechanism-specific guarantees. We analyze how subsampling affects the privacy of groups of multiple users.
  arXiv  Detail & Related papers  (2024-03-07T19:36:05Z)
• A Randomized Approach for Tight Privacy Accounting [63.67296945525791]
  We propose a new differential privacy paradigm called estimate-verify-release (EVR) EVR paradigm first estimates the privacy parameter of a mechanism, then verifies whether it meets this guarantee, and finally releases the query output. Our empirical evaluation shows the newly proposed EVR paradigm improves the utility-privacy tradeoff for privacy-preserving machine learning.
  arXiv  Detail & Related papers  (2023-04-17T00:38:01Z)
• Are We There Yet? Timing and Floating-Point Attacks on Differential Privacy Systems [18.396937775602808]
  We study two implementation flaws in the noise generation commonly used in differentially private (DP) systems. First we examine the Gaussian mechanism's susceptibility to a floating-point representation attack. Second we study discrete counterparts of the Laplace and Gaussian mechanisms that suffer from another side channel: a novel timing attack.
  arXiv  Detail & Related papers  (2021-12-10T02:57:01Z)
• Smoothed Differential Privacy [55.415581832037084]
  Differential privacy (DP) is a widely-accepted and widely-applied notion of privacy based on worst-case analysis. In this paper, we propose a natural extension of DP following the worst average-case idea behind the celebrated smoothed analysis. We prove that any discrete mechanism with sampling procedures is more private than what DP predicts, while many continuous mechanisms with sampling procedures are still non-private under smoothed DP.
  arXiv  Detail & Related papers  (2021-07-04T06:55:45Z)
• DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations [54.960853673256]
  We show that strong data augmentations, such as mixup and random additive noise, nullify poison attacks while enduring only a small accuracy trade-off. A rigorous analysis of DP-InstaHide shows that mixup does indeed have privacy advantages, and that training with k-way mixup provably yields at least k times stronger DP guarantees than a naive DP mechanism.
  arXiv  Detail & Related papers  (2021-03-02T23:07:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.