Empirical Evaluation of Structured Synthetic Data Privacy Metrics: Novel experimental framework

• URL: http://arxiv.org/abs/2512.16284v1
• Date: Thu, 18 Dec 2025 08:09:28 GMT
• Title: Empirical Evaluation of Structured Synthetic Data Privacy Metrics: Novel experimental framework
• Authors: Milton Nicolás Plasencia Palacios, Alexander Boudewijn, Sebastiano Saccani, Andrea Filippo Ferraris, Diana Sofronieva, Giuseppe D'Acquisto, Filiberto Brozzetti, Daniele Panfilo, Luca Bortolussi,
• Abstract summary: Synthetic data generation is gaining traction as a privacy enhancing technology.<n>The concept of data privacy remains elusive, making it challenging for practitioners to evaluate and benchmark the degree of privacy protection offered by synthetic data.
• Score: 34.56525983543448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Synthetic data generation is gaining traction as a privacy enhancing technology (PET). When properly generated, synthetic data preserve the analytic utility of real data while avoiding the retention of information that would allow the identification of specific individuals. However, the concept of data privacy remains elusive, making it challenging for practitioners to evaluate and benchmark the degree of privacy protection offered by synthetic data. In this paper, we propose a framework to empirically assess the efficacy of tabular synthetic data privacy quantification methods through controlled, deliberate risk insertion. To demonstrate this framework, we survey existing approaches to synthetic data privacy quantification and the related legal theory. We then apply the framework to the main privacy quantification methods with no-box threat models on publicly available datasets.

Related papers

• How to DP-fy Your Data: A Practical Guide to Generating Synthetic Data With Differential Privacy [52.00934156883483]
  Differential Privacy (DP) is a framework for reasoning about and limiting information leakage.<n>Differentially Private Synthetic data refers to synthetic data that preserves the overall trends of source data.
  arXiv  Detail & Related papers  (2025-12-02T21:14:39Z)
• Sensitivity, Specificity, and Consistency: A Tripartite Evaluation of Privacy Filters for Synthetic Data Generation [57.13635002340272]
  Post-hoc privacy filtering techniques have been proposed to remove samples containing personally identifiable information.<n>This work presents a rigorous evaluation of a filtering pipeline applied to chest X-ray synthesis.<n>We conclude that substantial advances in filter design are needed before these methods can be confidently deployed in sensitive applications.
  arXiv  Detail & Related papers  (2025-10-02T08:32:20Z)
• Aim High, Stay Private: Differentially Private Synthetic Data Enables Public Release of Behavioral Health Information with High Utility [2.1715431485081593]
  Differential Privacy (DP) provides formal guarantees against re-identification risks.<n>We generate DP synthetic data for Phase 1 of the Lived Experiences Measured Using Rings Study (LEMURS)<n>We evaluate the utility of the synthetic data using a framework informed by actual uses of the LEMURS dataset.
  arXiv  Detail & Related papers  (2025-06-30T15:58:34Z)
• A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage [77.83757117924995]
  We propose a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release.<n>Our approach shows that seemingly innocuous auxiliary information can be used to infer sensitive attributes like age or substance use history from sanitized data.
  arXiv  Detail & Related papers  (2025-04-28T01:16:27Z)
• A Consensus Privacy Metrics Framework for Synthetic Data [13.972528788909813]
  There is no consolidated standard for measuring privacy in synthetic data.<n>Our findings indicate that current similarity metrics fail to measure identity disclosure.<n>For differentially private synthetic data, a privacy budget other than close to zero was not considered interpretable.
  arXiv  Detail & Related papers  (2025-03-06T21:19:02Z)
• Tabular Data Synthesis with Differential Privacy: A Survey [24.500349285858597]
  Data sharing is a prerequisite for collaborative innovation, enabling organizations to leverage diverse datasets for deeper insights. Data synthesis tackles this by generating artificial datasets that preserve the statistical characteristics of real data. Differentially private data synthesis has emerged as a promising approach to privacy-aware data sharing.
  arXiv  Detail & Related papers  (2024-11-04T06:32:48Z)
• Mitigating the Privacy Issues in Retrieval-Augmented Generation (RAG) via Pure Synthetic Data [51.41288763521186]
  Retrieval-augmented generation (RAG) enhances the outputs of language models by integrating relevant information retrieved from external knowledge sources.<n>RAG systems may face severe privacy risks when retrieving private data.<n>We propose using synthetic data as a privacy-preserving alternative for the retrieval data.
  arXiv  Detail & Related papers  (2024-06-20T22:53:09Z)
• Auditing and Generating Synthetic Data with Controllable Trust Trade-offs [54.262044436203965]
  We introduce a holistic auditing framework that comprehensively evaluates synthetic datasets and AI models. It focuses on preventing bias and discrimination, ensures fidelity to the source data, assesses utility, robustness, and privacy preservation. We demonstrate the framework's effectiveness by auditing various generative models across diverse use cases.
  arXiv  Detail & Related papers  (2023-04-21T09:03:18Z)
• No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy" [75.98836424725437]
  New methods designed to preserve data privacy require careful scrutiny. Failure to preserve privacy is hard to detect, and yet can lead to catastrophic results when a system implementing a privacy-preserving'' method is attacked.
  arXiv  Detail & Related papers  (2022-09-29T17:50:23Z)
• Fidelity and Privacy of Synthetic Medical Data [0.0]
  The digitization of medical records ushered in a new era of big data to clinical science. The need to share individual-level medical data continues to grow, and has never been more urgent. enthusiasm for the use of big data has been tempered by a fully appropriate concern for patient autonomy and privacy.
  arXiv  Detail & Related papers  (2021-01-18T23:01:27Z)
• Really Useful Synthetic Data -- A Framework to Evaluate the Quality of Differentially Private Synthetic Data [2.538209532048867]
  Recent advances in generating synthetic data that allow to add principled ways of protecting privacy are a crucial step in sharing statistical information in a privacy preserving way. To further optimise the inherent trade-off between data privacy and data quality, it is necessary to think closely about the latter. We develop a framework to evaluate the quality of differentially private synthetic data from an applied researcher's perspective.
  arXiv  Detail & Related papers  (2020-04-16T16:24:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.