Generative Dataset Distillation using Min-Max Diffusion Model

• URL: http://arxiv.org/abs/2503.18626v1
• Date: Mon, 24 Mar 2025 12:41:40 GMT
• Title: Generative Dataset Distillation using Min-Max Diffusion Model
• Authors: Junqiao Fan, Yunjiao Zhou, Min Chang Jordan Ren, Jianfei Yang,
• Abstract summary: This paper addresses the problem of generative dataset distillation that utilizes generative models to synthesize images.<n>We leverage the popular diffusion model as the generator to compute a surrogate dataset, boosted by a min-max loss to control the dataset's diversity and representativeness during training.<n>We observe a critical trade-off between the number of image samples and the image quality controlled by the diffusion steps and propose Diffusion Step Reduction to achieve optimal performance.
• Score: 11.433714787534582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper, we address the problem of generative dataset distillation that utilizes generative models to synthesize images. The generator may produce any number of images under a preserved evaluation time. In this work, we leverage the popular diffusion model as the generator to compute a surrogate dataset, boosted by a min-max loss to control the dataset's diversity and representativeness during training. However, the diffusion model is time-consuming when generating images, as it requires an iterative generation process. We observe a critical trade-off between the number of image samples and the image quality controlled by the diffusion steps and propose Diffusion Step Reduction to achieve optimal performance. This paper details our comprehensive method and its performance. Our model achieved $2^{nd}$ place in the generative track of \href{https://www.dd-challenge.com/#/}{The First Dataset Distillation Challenge of ECCV2024}, demonstrating its superior performance.

Related papers

• D2C: Unlocking the Potential of Continuous Autoregressive Image Generation with Discrete Tokens [80.75893450536577]
  We propose D2C, a novel two-stage method to enhance model generation capacity. In the first stage, the discrete-valued tokens representing coarse-grained image features are sampled by employing a small discrete-valued generator. In the second stage, the continuous-valued tokens representing fine-grained image features are learned conditioned on the discrete token sequence.
  arXiv  Detail & Related papers  (2025-03-21T13:58:49Z)
• Context-guided Responsible Data Augmentation with Diffusion Models [29.41191005466334]
  We propose a text-to-image (T2I) data augmentation method, named DiffCoRe-Mix, that computes a set of generative counterparts for a training sample. To preserve key semantic axes, we also filter out undesired generative samples in our augmentation process. We extensively evaluate our technique on ImageNet-1K,Tiny ImageNet-200, CIFAR-100, Flowers102, CUB-Birds, Stanford Cars, and Caltech datasets.
  arXiv  Detail & Related papers  (2025-03-12T00:12:27Z)
• One-Step Diffusion Distillation through Score Implicit Matching [74.91234358410281]
  We present Score Implicit Matching (SIM) a new approach to distilling pre-trained diffusion models into single-step generator models. SIM shows strong empirical performances for one-step generators. By applying SIM to a leading transformer-based diffusion model, we distill a single-step generator for text-to-image generation.
  arXiv  Detail & Related papers  (2024-10-22T08:17:20Z)
• Generative Dataset Distillation Based on Diffusion Model [45.305885410046116]
  We propose a novel generative dataset distillation method based on Stable Diffusion. Specifically, we use the SDXL-Turbo model which can generate images at high speed and quality. We achieved third place in the generative track of the ECCV 2024 DD Challenge.
  arXiv  Detail & Related papers  (2024-08-16T08:52:02Z)
• EM Distillation for One-step Diffusion Models [65.57766773137068]
  We propose a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of quality.<n>We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process.
  arXiv  Detail & Related papers  (2024-05-27T05:55:22Z)
• Improved Distribution Matching Distillation for Fast Image Synthesis [54.72356560597428]
  We introduce DMD2, a set of techniques that lift this limitation and improve DMD training. First, we eliminate the regression loss and the need for expensive dataset construction. Second, we integrate a GAN loss into the distillation procedure, discriminating between generated samples and real images.
  arXiv  Detail & Related papers  (2024-05-23T17:59:49Z)
• Distilling Diffusion Models into Conditional GANs [90.76040478677609]
  We distill a complex multistep diffusion model into a single-step conditional GAN student model. For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space. We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
  arXiv  Detail & Related papers  (2024-05-09T17:59:40Z)
• Active Generation for Image Classification [45.93535669217115]
  We propose to address the efficiency of image generation by focusing on the specific needs and characteristics of the model. With a central tenet of active learning, our method, named ActGen, takes a training-aware approach to image generation.
  arXiv  Detail & Related papers  (2024-03-11T08:45:31Z)
• Efficient Dataset Distillation via Minimax Diffusion [24.805804922949832]
  We present a theoretical model of the process as hierarchical diffusion control demonstrating the flexibility of the diffusion process to target these criteria. Under the 100-IPC setting on ImageWoof, our method requires less than one-twentieth the distillation time of previous methods, yet yields even better performance.
  arXiv  Detail & Related papers  (2023-11-27T04:22:48Z)
• BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping [64.54271680071373]
  Diffusion models have demonstrated excellent potential for generating diverse images. Knowledge distillation has been recently proposed as a remedy that can reduce the number of inference steps to one or a few. We present a novel technique called BOOT, that overcomes limitations with an efficient data-free distillation algorithm.
  arXiv  Detail & Related papers  (2023-06-08T20:30:55Z)
• Nested Diffusion Processes for Anytime Image Generation [38.84966342097197]
  We propose an anytime diffusion-based method that can generate viable images when stopped at arbitrary times before completion. In experiments on ImageNet and Stable Diffusion-based text-to-image generation, we show, both qualitatively and quantitatively, that our method's intermediate generation quality greatly exceeds that of the original diffusion model.
  arXiv  Detail & Related papers  (2023-05-30T14:28:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.