Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data

• URL: http://arxiv.org/abs/2504.14368v1
• Date: Sat, 19 Apr 2025 17:55:10 GMT
• Title: Do You Really Need Public Data? Surrogate Public Data for Differential Privacy on Tabular Data
• Authors: Shlomi Hod, Lucas Rosenblatt, Julia Stoyanovich,
• Abstract summary: This work introduces the notion of "surrogate" public data, which consume no privacy loss budget and are constructed solely from publicly available schema or metadata.<n>We automate the process of generating surrogate public data with large language models (LLMs)<n>In particular, we propose two methods: direct record generation as CSV files, and automated structural causal model (SCM) construction for sampling records.
• Score: 10.1687640711587
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Differentially private (DP) machine learning often relies on the availability of public data for tasks like privacy-utility trade-off estimation, hyperparameter tuning, and pretraining. While public data assumptions may be reasonable in text and image domains, they are less likely to hold for tabular data due to tabular data heterogeneity across domains. We propose leveraging powerful priors to address this limitation; specifically, we synthesize realistic tabular data directly from schema-level specifications - such as variable names, types, and permissible ranges - without ever accessing sensitive records. To that end, this work introduces the notion of "surrogate" public data - datasets generated independently of sensitive data, which consume no privacy loss budget and are constructed solely from publicly available schema or metadata. Surrogate public data are intended to encode plausible statistical assumptions (informed by publicly available information) into a dataset with many downstream uses in private mechanisms. We automate the process of generating surrogate public data with large language models (LLMs); in particular, we propose two methods: direct record generation as CSV files, and automated structural causal model (SCM) construction for sampling records. Through extensive experiments, we demonstrate that surrogate public tabular data can effectively replace traditional public data when pretraining differentially private tabular classifiers. To a lesser extent, surrogate public data are also useful for hyperparameter tuning of DP synthetic data generators, and for estimating the privacy-utility tradeoff.

Related papers

• Tabular Data Adapters: Improving Outlier Detection for Unlabeled Private Data [12.092540602813333]
  We introduce Tabular Data Adapters (TDA), a novel method for generating soft labels for unlabeled data in outlier detection tasks. Our approach offers a scalable, efficient, and cost-effective solution, to bridge the gap between public research models and real-world industrial applications.
  arXiv  Detail & Related papers  (2025-04-29T15:38:43Z)
• Leveraging Vertical Public-Private Split for Improved Synthetic Data Generation [9.819636361032256]
  Differentially Private Synthetic Data Generation is a key enabler of private and secure data sharing.<n>Recent literature has explored scenarios where a small amount of public data is used to help enhance the quality of synthetic data.<n>We propose a novel framework that adapts horizontal public-assisted methods into the vertical setting.
  arXiv  Detail & Related papers  (2025-04-15T08:59:03Z)
• Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs [20.774525687291167]
  We propose a novel framework for generating privacy-preserving synthetic data without extensive prompt engineering or billion-scale finetuning.<n> CTCL pretrains a lightweight 140M conditional generator and a clustering-based topic model on large-scale public data.<n>To further adapt to the private domain, the generator is DP finetuned on private data for fine-grained textual information, while the topic model extracts a DP histogram.
  arXiv  Detail & Related papers  (2025-03-16T04:00:32Z)
• Private prediction for large-scale synthetic text generation [28.488459921169905]
  We present an approach for generating differentially private synthetic text using large language models (LLMs) In the private prediction framework, we only require the output synthetic data to satisfy differential privacy guarantees.
  arXiv  Detail & Related papers  (2024-07-16T18:28:40Z)
• Joint Selection: Adaptively Incorporating Public Information for Private Synthetic Data [13.56146208014469]
  We develop the mechanism jam-pgm, which expands the adaptive measurements framework to jointly select between measuring public data and private data. We show that jam-pgm is able to outperform both publicly assisted and non publicly assisted synthetic data generation mechanisms even when the public data distribution is biased.
  arXiv  Detail & Related papers  (2024-03-12T16:34:07Z)
• Privacy Amplification for the Gaussian Mechanism via Bounded Support [64.86780616066575]
  Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. We propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting.
  arXiv  Detail & Related papers  (2024-03-07T21:22:07Z)
• PrivacyMind: Large Language Models Can Be Contextual Privacy Protection Learners [81.571305826793]
  We introduce Contextual Privacy Protection Language Models (PrivacyMind) Our work offers a theoretical analysis for model design and benchmarks various techniques. In particular, instruction tuning with both positive and negative examples stands out as a promising method.
  arXiv  Detail & Related papers  (2023-10-03T22:37:01Z)
• A Unified View of Differentially Private Deep Generative Modeling [60.72161965018005]
  Data with privacy concerns comes with stringent regulations that frequently prohibited data access and data sharing. Overcoming these obstacles is key for technological progress in many real-world application scenarios that involve privacy sensitive data. Differentially private (DP) data publishing provides a compelling solution, where only a sanitized form of the data is publicly released.
  arXiv  Detail & Related papers  (2023-09-27T14:38:16Z)
• Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining [75.25943383604266]
  We question whether the use of large Web-scraped datasets should be viewed as differential-privacy-preserving. We caution that publicizing these models pretrained on Web data as "private" could lead to harm and erode the public's trust in differential privacy as a meaningful definition of privacy. We conclude by discussing potential paths forward for the field of private learning, as public pretraining becomes more popular and powerful.
  arXiv  Detail & Related papers  (2022-12-13T10:41:12Z)
• Private Set Generation with Discriminative Information [63.851085173614]
  Differentially private data generation is a promising solution to the data privacy challenge. Existing private generative models are struggling with the utility of synthetic samples. We introduce a simple yet effective method that greatly improves the sample utility of state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-07T10:02:55Z)
• DP2-Pub: Differentially Private High-Dimensional Data Publication with Invariant Post Randomization [58.155151571362914]
  We propose a differentially private high-dimensional data publication mechanism (DP2-Pub) that runs in two phases. splitting attributes into several low-dimensional clusters with high intra-cluster cohesion and low inter-cluster coupling helps obtain a reasonable privacy budget. We also extend our DP2-Pub mechanism to the scenario with a semi-honest server which satisfies local differential privacy.
  arXiv  Detail & Related papers  (2022-08-24T17:52:43Z)
• Generating private data with user customization [9.415164800448853]
  Mobile devices can produce and store large amounts of data that can enhance machine learning models. However, this data may contain private information specific to the data owner that prevents the release of the data. We want to reduce the correlation between user-specific private information and the data while retaining the useful information.
  arXiv  Detail & Related papers  (2020-12-02T19:13:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.