Turning Black Box into White Box: Dataset Distillation Leaks

• URL: http://arxiv.org/abs/2603.01053v1
• Date: Sun, 01 Mar 2026 11:21:28 GMT
• Title: Turning Black Box into White Box: Dataset Distillation Leaks
• Authors: Huajie Chen, Tianqing Zhu, Yuchen Zhong, Yang Zhang, Shang Wang, Feng He, Lefeng Zhang, Jialiang Shen, Minghao Wang, Wanlei Zhou,
• Abstract summary: We show that existing distillation methods can cause severe privacy leakage because synthetic datasets implicitly encode the weight trajectories of the distilled model.<n>We introduce the Information Revelation Attack (IRA) against state-of-the-art distillation techniques.<n>Experiments show that IRA accurately predicts both the distillation algorithm and model architecture, and can successfully infer membership and recover sensitive samples from the real dataset.
• Score: 25.648032166889465
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Dataset distillation compresses a large real dataset into a small synthetic one, enabling models trained on the synthetic data to achieve performance comparable to those trained on the real data. Although synthetic datasets are assumed to be privacy-preserving, we show that existing distillation methods can cause severe privacy leakage because synthetic datasets implicitly encode the weight trajectories of the distilled model, they become over-informative and exploitable by adversaries. To expose this risk, we introduce the Information Revelation Attack (IRA) against state-of-the-art distillation techniques. Experiments show that IRA accurately predicts both the distillation algorithm and model architecture, and can successfully infer membership and recover sensitive samples from the real dataset.

Related papers

• Osmosis Distillation: Model Hijacking with the Fewest Samples [27.65508058446939]
  A non-negligible security threat remains undiscovered in transfer learning using synthetic datasets generated by dataset distillation methods.<n>We propose Osmosis Distillation (OD) attack, a novel model hijacking strategy that targets deep learning models using the fewest samples.<n>We argue that awareness of using third-party synthetic datasets in transfer learning must be raised.
  arXiv  Detail & Related papers  (2026-03-05T06:34:06Z)
• Robust Dataset Distillation by Matching Adversarial Trajectories [21.52323435014135]
  We introduce the task of robust dataset distillation", a novel paradigm that embeds adversarial robustness into synthetic datasets during the distillation process.<n>We propose Matching Adversarial Trajectories (MAT), a method that integrates adversarial training into trajectory-based dataset distillation.<n>MAT incorporates adversarial samples during trajectory generation to obtain robust training trajectories, which are then used to guide the distillation process.
  arXiv  Detail & Related papers  (2025-03-15T10:02:38Z)
• Dark Distillation: Backdooring Distilled Datasets without Accessing Raw Data [48.69361050757504]
  This work is the first to address a more realistic and concerning threat: attackers may intercept the dataset distribution process, inject backdoors into the distilled datasets, and redistribute them to users.<n>While distilled datasets were previously considered resistant to backdoor attacks, we demonstrate that they remain vulnerable to such attacks.<n>Our attack method is efficient, capable of a malicious distilled dataset in under one minute in certain cases.
  arXiv  Detail & Related papers  (2025-02-06T17:14:17Z)
• What is Dataset Distillation Learning? [32.99890244958794]
  We study the behavior, representativeness, and point-wise information content of distilled data. We reveal distilled data cannot serve as a substitute for real data during training. We provide an framework for interpreting distilled data and reveal that individual distilled data points contain meaningful semantic information.
  arXiv  Detail & Related papers  (2024-06-06T17:28:56Z)
• Importance-Aware Adaptive Dataset Distillation [53.79746115426363]
  Development of deep learning models is enabled by the availability of large-scale datasets. dataset distillation aims to synthesize a compact dataset that retains the essential information from the large original dataset. We propose an importance-aware adaptive dataset distillation (IADD) method that can improve distillation performance.
  arXiv  Detail & Related papers  (2024-01-29T03:29:39Z)
• Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation [96.92250565207017]
  We study the data efficiency and selection for the dataset distillation task. By re-formulating the dynamics of distillation, we provide insight into the inherent redundancy in the real dataset. We find the most contributing samples based on their causal effects on the distillation.
  arXiv  Detail & Related papers  (2023-05-28T06:53:41Z)
• Generalizing Dataset Distillation via Deep Generative Prior [75.9031209877651]
  We propose to distill an entire dataset's knowledge into a few synthetic images. The idea is to synthesize a small number of synthetic data points that, when given to a learning algorithm as training data, result in a model approximating one trained on the original data. We present a new optimization algorithm that distills a large number of images into a few intermediate feature vectors in the generative model's latent space.
  arXiv  Detail & Related papers  (2023-05-02T17:59:31Z)
• Backdoor Attacks Against Dataset Distillation [24.39067295054253]
  This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases.
  arXiv  Detail & Related papers  (2023-01-03T16:58:34Z)
• Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation [151.70234052015948]
  We propose a novel approach that encourages the optimization algorithm to seek a flat trajectory. We show that the weights trained on synthetic data are robust against the accumulated errors perturbations with the regularization towards the flat trajectory. Our method, called Flat Trajectory Distillation (FTD), is shown to boost the performance of gradient-matching methods by up to 4.7%.
  arXiv  Detail & Related papers  (2022-11-20T15:49:11Z)
• Dataset Distillation by Matching Training Trajectories [75.9031209877651]
  We propose a new formulation that optimize our distilled data to guide networks to a similar state as those trained on real data. Given a network, we train it for several iterations on our distilled data and optimize the distilled data with respect to the distance between the synthetically trained parameters and the parameters trained on real data. Our method handily outperforms existing methods and also allows us to distill higher-resolution visual data.
  arXiv  Detail & Related papers  (2022-03-22T17:58:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.