DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillation

• URL: http://arxiv.org/abs/2403.13322v2
• Date: Mon, 27 May 2024 13:11:08 GMT
• Title: DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillation
• Authors: Yifan Wu, Jiawei Du, Ping Liu, Yuewei Lin, Wenqing Cheng, Wei Xu,
• Abstract summary: We introduce a comprehensive benchmark that is the most extensive to date for evaluating the adversarial robustness of distilled datasets in a unified way. Our benchmark significantly expands upon prior efforts by incorporating the latest advancements such as TESLA and SRe2L. We also discovered that incorporating distilled data into the training batches of the original dataset can yield to improvement of robustness.
• Score: 25.7548771762807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dataset distillation is an advanced technique aimed at compressing datasets into significantly smaller counterparts, while preserving formidable training performance. Significant efforts have been devoted to promote evaluation accuracy under limited compression ratio while overlooked the robustness of distilled dataset. In this work, we introduce a comprehensive benchmark that, to the best of our knowledge, is the most extensive to date for evaluating the adversarial robustness of distilled datasets in a unified way. Our benchmark significantly expands upon prior efforts by incorporating a wider range of dataset distillation methods, including the latest advancements such as TESLA and SRe2L, a diverse array of adversarial attack methods, and evaluations across a broader and more extensive collection of datasets such as ImageNet-1K. Moreover, we assessed the robustness of these distilled datasets against representative adversarial attack algorithms like PGD and AutoAttack, while exploring their resilience from a frequency perspective. We also discovered that incorporating distilled data into the training batches of the original dataset can yield to improvement of robustness.

Related papers

• Practical Dataset Distillation Based on Deep Support Vectors [27.16222034423108]
  In this paper, we focus on dataset distillation in practical scenarios with access to only a fraction of the entire dataset. We introduce a novel distillation method that augments the conventional process by incorporating general model knowledge via the addition of Deep KKT (DKKT) loss. In practical settings, our approach showed improved performance compared to the baseline distribution matching distillation method on the CIFAR-10 dataset.
  arXiv  Detail & Related papers  (2024-05-01T06:41:27Z)
• Towards Adversarially Robust Dataset Distillation by Curvature Regularization [11.463315774971857]
  We study how to embed adversarial robustness in distilled datasets, so that models trained on these datasets maintain the high accuracy and acquire better adversarial robustness. We propose a new method that achieves this goal by incorporating curvature regularization into the distillation process with much less computational overhead than standard adversarial training.
  arXiv  Detail & Related papers  (2024-03-15T06:31:03Z)
• Importance-Aware Adaptive Dataset Distillation [53.79746115426363]
  Development of deep learning models is enabled by the availability of large-scale datasets. dataset distillation aims to synthesize a compact dataset that retains the essential information from the large original dataset. We propose an importance-aware adaptive dataset distillation (IADD) method that can improve distillation performance.
  arXiv  Detail & Related papers  (2024-01-29T03:29:39Z)
• Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching [7.908244841289913]
  The expenses involved in training state-of-the-art deep hashing retrieval models have witnessed an increase. The state-of-the-art dataset distillation methods can not expand to all deep hashing retrieval methods. We propose an efficient condensation framework that addresses these limitations by matching the feature-embedding between synthetic set and real set.
  arXiv  Detail & Related papers  (2023-05-29T13:23:55Z)
• Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation [96.92250565207017]
  We study the data efficiency and selection for the dataset distillation task. By re-formulating the dynamics of distillation, we provide insight into the inherent redundancy in the real dataset. We find the most contributing samples based on their causal effects on the distillation.
  arXiv  Detail & Related papers  (2023-05-28T06:53:41Z)
• A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairness [8.432686179800543]
  We conduct extensive experiments to evaluate current state-of-the-art dataset distillation methods. We successfully use membership inference attacks to show that privacy risks still remain. This work offers a large-scale benchmarking framework for dataset distillation evaluation.
  arXiv  Detail & Related papers  (2023-05-05T08:19:27Z)
• Dataset Distillation: A Comprehensive Review [76.26276286545284]
  dataset distillation (DD) aims to derive a much smaller dataset containing synthetic samples, based on which the trained models yield performance comparable with those trained on the original dataset. This paper gives a comprehensive review and summary of recent advances in DD and its application.
  arXiv  Detail & Related papers  (2023-01-17T17:03:28Z)
• A Comprehensive Survey of Dataset Distillation [73.15482472726555]
  It has become challenging to handle the unlimited growth of data with limited computing power. Deep learning technology has developed unprecedentedly in the last decade. This paper provides a holistic understanding of dataset distillation from multiple aspects.
  arXiv  Detail & Related papers  (2023-01-13T15:11:38Z)
• Dataset Distillation via Factorization [58.8114016318593]
  We introduce a emphdataset factorization approach, termed emphHaBa, which is a plug-and-play strategy portable to any existing dataset distillation (DD) baseline. emphHaBa explores decomposing a dataset into two components: data emphHallucination networks and emphBases. Our method can yield significant improvement on downstream classification tasks compared with previous state of the arts, while reducing the total number of compressed parameters by up to 65%.
  arXiv  Detail & Related papers  (2022-10-30T08:36:19Z)
• DC-BENCH: Dataset Condensation Benchmark [79.18718490863908]
  This work provides the first large-scale standardized benchmark on dataset condensation. It consists of a suite of evaluations to comprehensively reflect the generability and effectiveness of condensation methods. The benchmark library is open-sourced to facilitate future research and application.
  arXiv  Detail & Related papers  (2022-07-20T03:54:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.