Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learning

• URL: http://arxiv.org/abs/2603.00811v1
• Date: Sat, 28 Feb 2026 21:14:01 GMT
• Title: Curation Leaks: Membership Inference Attacks against Data Curation for Machine Learning
• Authors: Dariush Wahdany, Matthew Jagielski, Adam Dziedzic, Franziska Boenisch,
• Abstract summary: We show that without further protection, curation pipelines can still leak private information.<n>We demonstrate that each stage reveals information about the private dataset and that even models trained exclusively on curated public data leak membership information about the private data that guided curation.
• Score: 36.4616907441652
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In machine learning, curation is used to select the most valuable data for improving both model accuracy and computational efficiency. Recently, curation has also been explored as a solution for private machine learning: rather than training directly on sensitive data, which is known to leak information through model predictions, the private data is used only to guide the selection of useful public data. The resulting model is then trained solely on curated public data. It is tempting to assume that such a model is privacy-preserving because it has never seen the private data. Yet, we show that without further protection, curation pipelines can still leak private information. Specifically, we introduce novel attacks against popular curation methods, targeting every major step: the computation of curation scores, the selection of the curated subset, and the final trained model. We demonstrate that each stage reveals information about the private dataset and that even models trained exclusively on curated public data leak membership information about the private data that guided curation. These findings highlight the previously overlooked inherent privacy risks of data curation and show that privacy assessment must extend beyond the training procedure to include the data selection process. Our differentially private adaptations of curation methods effectively mitigate leakage, indicating that formal privacy guarantees for curation are a promising direction.

Related papers

• A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage [77.83757117924995]
  We propose a new framework that evaluates re-identification attacks to quantify individual privacy risks upon data release.<n>Our approach shows that seemingly innocuous auxiliary information can be used to infer sensitive attributes like age or substance use history from sanitized data.
  arXiv  Detail & Related papers  (2025-04-28T01:16:27Z)
• Pseudo-Probability Unlearning: Towards Efficient and Privacy-Preserving Machine Unlearning [59.29849532966454]
  We propose PseudoProbability Unlearning (PPU), a novel method that enables models to forget data to adhere to privacy-preserving manner. Our method achieves over 20% improvements in forgetting error compared to the state-of-the-art.
  arXiv  Detail & Related papers  (2024-11-04T21:27:06Z)
• Simulation-based Bayesian Inference from Privacy Protected Data [0.0]
  We propose simulation-based inference methods from privacy-protected datasets.<n>We illustrate our methods on discrete time-series data under an infectious disease model and with ordinary linear regression models.
  arXiv  Detail & Related papers  (2023-10-19T14:34:17Z)
• Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining [75.25943383604266]
  We question whether the use of large Web-scraped datasets should be viewed as differential-privacy-preserving. We caution that publicizing these models pretrained on Web data as "private" could lead to harm and erode the public's trust in differential privacy as a meaningful definition of privacy. We conclude by discussing potential paths forward for the field of private learning, as public pretraining becomes more popular and powerful.
  arXiv  Detail & Related papers  (2022-12-13T10:41:12Z)
• Certified Data Removal in Sum-Product Networks [78.27542864367821]
  Deleting the collected data is often insufficient to guarantee data privacy. UnlearnSPN is an algorithm that removes the influence of single data points from a trained sum-product network.
  arXiv  Detail & Related papers  (2022-10-04T08:22:37Z)
• SecretGen: Privacy Recovery on Pre-Trained Models via Distribution Discrimination [17.916489394284284]
  We propose a novel private data reconstruction framework, SecretGen, to effectively recover private information. SecretGen does not require prior knowledge about true class prediction. We show that SecretGen is able to recover private data with similar performance compared with the ones that leverage such prior knowledge.
  arXiv  Detail & Related papers  (2022-07-25T15:35:07Z)
• Personalized PATE: Differential Privacy for Machine Learning with Individual Privacy Guarantees [1.2691047660244335]
  We propose three novel methods to support training an ML model with different personalized privacy guarantees within the training data. Our experiments show that our personalized privacy methods yield higher accuracy models than the non-personalized baseline.
  arXiv  Detail & Related papers  (2022-02-21T20:16:27Z)
• Towards a Data Privacy-Predictive Performance Trade-off [2.580765958706854]
  We evaluate the existence of a trade-off between data privacy and predictive performance in classification tasks. Unlike previous literature, we confirm that the higher the level of privacy, the higher the impact on predictive performance.
  arXiv  Detail & Related papers  (2022-01-13T21:48:51Z)
• Chasing Your Long Tails: Differentially Private Prediction in Health Care Settings [34.26542589537452]
  Methods for differentially private (DP) learning provide a general-purpose approach to learn models with privacy guarantees. Modern methods for DP learning ensure privacy through mechanisms that censor information judged as too unique. We use state-of-the-art methods for DP learning to train privacy-preserving models in clinical prediction tasks.
  arXiv  Detail & Related papers  (2020-10-13T19:56:37Z)
• TIPRDC: Task-Independent Privacy-Respecting Data Crowdsourcing Framework for Deep Learning with Anonymized Intermediate Representations [49.20701800683092]
  We present TIPRDC, a task-independent privacy-respecting data crowdsourcing framework with anonymized intermediate representation. The goal of this framework is to learn a feature extractor that can hide the privacy information from the intermediate representations; while maximally retaining the original information embedded in the raw data for the data collector to accomplish unknown learning tasks.
  arXiv  Detail & Related papers  (2020-05-23T06:21:26Z)
• Utility-aware Privacy-preserving Data Releasing [7.462336024223669]
  We propose a two-step perturbation-based privacy-preserving data releasing framework. First, certain predefined privacy and utility problems are learned from the public domain data. We then leverage the learned knowledge to precisely perturb the data owners' data into privatized data.
  arXiv  Detail & Related papers  (2020-05-09T05:32:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.