Subgroup Membership Inference Audits of Differentially Private Synthetic Text

• URL: http://arxiv.org/abs/2609.09848v1
• Date: Wed, 09 Sep 2026 07:56:24 GMT
• Title: Subgroup Membership Inference Audits of Differentially Private Synthetic Text
• Abstract summary: Even when the worst-case privacy leakage of such releases is bounded by means of differential privacy (DP), in practice a residual risk remains.<n>This paper defines a subgroup-targeted membership inference game in which the target pool is an explicit parameter, and instantiates it with an audit of 32 proxies.<n>The audit shows that synthetic releases leak subgroup membership and that prior attacks systematically underestimate this leakage.
• Score: 28.947903737268916
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Synthetic data releases are increasingly proposed in the literature as a means of sharing realistic data replicas in lieu of sensitive private datasets. Even when the worst-case privacy leakage of such releases is bounded by means of differential privacy (DP), in practice a residual risk remains. Membership inference attack (MIA) audits are conducted to empirically quantify this risk. However, existing methods only measure average-case risk for randomly drawn records, which might conceal the risk to vulnerable subgroups. To highlight this issue, we define a subgroup-targeted membership inference game in which the target pool is an explicit parameter, and instantiate it with an audit of 32 proxies under three scenarios with different levels of attacker knowledge, across four datasets, three generators (DP-SGD fine-tuning, API-based prompting, and activation steering), and five privacy budgets. The audit shows that synthetic releases leak subgroup membership and that prior attacks systematically underestimate this leakage. DP is effective at the aggregate level: it substantially reduces average leakage at every budget we test. Three observations temper this picture. First, the remaining leakage is concentrated rather than spread out: under DP, a tenth of the records carries roughly 40% of it. Second, the protection DP delivers in practice is uneven: within its worst-case guarantee, the noise removes more of the measured leakage from random records than from high-risk ones---and a merged-pool audit that scores both record types against shared negatives confirms this at the record level. Third, \emph{which} records leak proves to be a property of the release mechanism rather than of the record alone, so record-level risk cannot be assessed independently of the release.

Related papers

• DiffSafeMerge: Mitigating Backdoor Inheritance in Diffusion Model Merging [12.412925762132284]
  Mitigation is difficult without knowing the compromised source, trigger, or target.<n>We introduce DiffSafeMerge (DSM), which uses a small unlabeled clean set and fixed, attack-agnostic stress probes to score source blocks.
  arXiv  Detail & Related papers  (2026-08-10T11:20:35Z)
• Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents [0.0]
  Traditional taint tracking permanently taints an agent's context upon reading unvetted data.<n>We present APPA, an IFC framework that resolves this usability bottleneck through engine-managed context branching.<n>We evaluate APPA on a multi-turn tool-chaining benchmark across four models.
  arXiv  Detail & Related papers  (2026-07-27T16:19:45Z)
• Phantoms and Disclosures: a Causal Framework for Auditing Synthetic Data [56.86147283213652]
  We present a customizable empirical auditing framework designed to detect and explain data disclosures.<n>Our framework introduces a mechanism to distinguish between "true disclosures"-where the system directly reproduces a user's information-and "phantom disclosures"-where the system incidentally generates a user's data.
  arXiv  Detail & Related papers  (2026-06-15T16:54:02Z)
• OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents [32.5466552510287]
  Leakage is cumulative, as individually innocuous releases accumulate across honest-but-curious or colluding sinks.<n>We present OCELOT, a runtime mediator that budgets how much an adversary's belief about a secret may improve.<n>OCELOT attains significantly lower leakage at higher task utility, resists adaptive injection, jailbreak, cumulative inference, and sink collusion.
  arXiv  Detail & Related papers  (2026-06-10T17:13:35Z)
• Risk-Equalized Differentially Private Synthetic Data: Protecting Outliers by Controlling Record-Level Influence [9.016539021471845]
  This paper introduces risk-equalized DP synthesis, a framework that prioritizes protection for high-risk records.<n>Under Gaussian mechanisms, a record's privacy loss is proportional to its influence on the output.<n>Experiments show that risk-weighting substantially reduces membership inference success against high-outlierness records.
  arXiv  Detail & Related papers  (2026-02-10T19:22:46Z)
• Perfectly-Private Analog Secure Aggregation in Federated Learning [51.61616734974475]
  In federated learning, multiple parties train models locally and share their parameters with a central server, which aggregates them to update a global model.<n>In this paper, a novel secure parameter aggregation method is proposed that employs the torus rather than a finite field.
  arXiv  Detail & Related papers  (2025-09-10T15:22:40Z)
• Benchmarking Fraud Detectors on Private Graph Data [70.4654745317714]
  Currently, many types of fraud are managed in part by automated detection algorithms that operate over graphs.<n>We consider the scenario where a data holder wishes to outsource development of fraud detectors to third parties.<n>Third parties submit their fraud detectors to the data holder, who evaluates these algorithms on a private dataset and then publicly communicates the results.<n>We propose a realistic privacy attack on this system that allows an adversary to de-anonymize individuals' data based only on the evaluation results.
  arXiv  Detail & Related papers  (2025-07-30T03:20:15Z)
• CBW: Towards Dataset Ownership Verification for Speaker Verification via Clustering-based Backdoor Watermarking [85.68235482145091]
  Large-scale speech datasets have become valuable intellectual property.<n>We propose a novel dataset ownership verification method.<n>Our approach introduces a clustering-based backdoor watermark (CBW)<n>We conduct extensive experiments on benchmark datasets, verifying the effectiveness and robustness of our method against potential adaptive attacks.
  arXiv  Detail & Related papers  (2025-03-02T02:02:57Z)
• Privacy Amplification for the Gaussian Mechanism via Bounded Support [64.86780616066575]
  Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. We propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting.
  arXiv  Detail & Related papers  (2024-03-07T21:22:07Z)
• Analyzing Leakage of Personally Identifiable Information in Language Models [13.467340359030855]
  Language Models (LMs) have been shown to leak information about training data through sentence-level membership inference and reconstruction attacks. Scrubbing techniques reduce but do not prevent the risk of PII leakage. It is unclear to which extent algorithmic defenses such as differential privacy, designed to guarantee user-level privacy, prevent PII disclosure.
  arXiv  Detail & Related papers  (2023-02-01T16:04:48Z)
• Improved Certified Defenses against Data Poisoning with (Deterministic) Finite Aggregation [122.83280749890078]
  We propose an improved certified defense against general poisoning attacks, namely Finite Aggregation. In contrast to DPA, which directly splits the training set into disjoint subsets, our method first splits the training set into smaller disjoint subsets. We offer an alternative view of our method, bridging the designs of deterministic and aggregation-based certified defenses.
  arXiv  Detail & Related papers  (2022-02-05T20:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.