PRISM: Precision-Recall Informed Data-Free Knowledge Distillation via Generative Diffusion

• URL: http://arxiv.org/abs/2509.16897v1
• Date: Sun, 21 Sep 2025 03:16:07 GMT
• Title: PRISM: Precision-Recall Informed Data-Free Knowledge Distillation via Generative Diffusion
• Authors: Xuewan He, Jielei Wang, Zihan Cheng, Yuchen Su, Shiyue Huang, Guoming Lu,
• Abstract summary: Data-free knowledge distillation (DFKD) transfers knowledge from a teacher to a student without access to the real in-distribution (ID) data.<n>While existing methods perform well on small-scale images, they suffer from mode collapse when synthesizing large-scale images.<n>We propose PRISM, a precision-recall informed method for synthesizing photorealistic images.
• Score: 4.591973713524844
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Data-free knowledge distillation (DFKD) transfers knowledge from a teacher to a student without access to the real in-distribution (ID) data. While existing methods perform well on small-scale images, they suffer from mode collapse when synthesizing large-scale images, resulting in limited knowledge transfer. Recently, leveraging advanced generative models to synthesize photorealistic images has emerged as a promising alternative. Nevertheless, directly using off-the-shelf diffusion to generate datasets faces the precision-recall challenges: 1) ensuring synthetic data aligns with the real distribution, and 2) ensuring coverage of the real ID manifold. In response, we propose PRISM, a precision-recall informed synthesis method. Specifically, we introduce Energy-guided Distribution Alignment to avoid the generation of out-of-distribution samples, and design the Diversified Prompt Engineering to enhance coverage of the real ID manifold. Extensive experiments on various large-scale image datasets demonstrate the superiority of PRISM. Moreover, we demonstrate that models trained with PRISM exhibit strong domain generalization.

Related papers

• Edit2Perceive: Image Editing Diffusion Models Are Strong Dense Perceivers [55.15722080205737]
  Edit2Perceive is a unified diffusion framework that adapts editing models for depth, normal, and matting.<n>Our single-step deterministic inference yields up to faster runtime while training on relatively small datasets.
  arXiv  Detail & Related papers  (2025-11-24T01:13:51Z)
• Enhancing Diffusion-based Dataset Distillation via Adversary-Guided Curriculum Sampling [22.21686398518648]
  Adversary-guided Curriculum Sampling (ACS) partitions distilled dataset into multiple curricula.<n>ACS guides diffusion sampling process by an adversarial loss to challenge a discriminator trained on sampled images.<n>ACS achieves substantial improvements of 4.1% on Imagewoof and 2.1% on ImageNet-1k over the state-of-the-art.
  arXiv  Detail & Related papers  (2025-08-02T08:48:32Z)
• When Model Knowledge meets Diffusion Model: Diffusion-assisted Data-free Image Synthesis with Alignment of Domain and Class [18.81528537866941]
  Open-source pre-trained models hold great potential for diverse applications, but their utility declines when their training data is unavailable.<n>Data-Free Image Synthesis (DFIS) aims to generate images that approximate the learned data distribution of a pre-trained model without accessing the original data.<n>DDIS is the first Diffusion-assisted Data-free Image Synthesis method that leverages a text-to-image diffusion model as a powerful image prior.
  arXiv  Detail & Related papers  (2025-06-18T11:51:40Z)
• Boosting Zero-shot Stereo Matching using Large-scale Mixed Images Sources in the Real World [8.56549004133167]
  Stereo matching methods rely on dense pixel-wise ground truth labels.<n>The scarcity of labeled data and domain gaps between synthetic and real-world images pose notable challenges.<n>We propose a novel framework, textbfBooSTer, that leverages both vision foundation models and large-scale mixed image sources.
  arXiv  Detail & Related papers  (2025-05-13T14:24:38Z)
• Large-Scale Data-Free Knowledge Distillation for ImageNet via Multi-Resolution Data Generation [53.95204595640208]
  Data-Free Knowledge Distillation (DFKD) is an advanced technique that enables knowledge transfer from a teacher model to a student model without relying on original training data. Previous approaches have generated synthetic images at high resolutions without leveraging information from real images. MUSE generates images at lower resolutions while using Class Activation Maps (CAMs) to ensure that the generated images retain critical, class-specific features.
  arXiv  Detail & Related papers  (2024-11-26T02:23:31Z)
• One Step Diffusion-based Super-Resolution with Time-Aware Distillation [60.262651082672235]
  Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. Recent techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. We propose a time-aware diffusion distillation method, named TAD-SR, to accomplish effective and efficient image super-resolution.
  arXiv  Detail & Related papers  (2024-08-14T11:47:22Z)
• Contrasting Deepfakes Diffusion via Contrastive Learning and Global-Local Similarities [88.398085358514]
  Contrastive Deepfake Embeddings (CoDE) is a novel embedding space specifically designed for deepfake detection. CoDE is trained via contrastive learning by additionally enforcing global-local similarities.
  arXiv  Detail & Related papers  (2024-07-29T18:00:10Z)
• Rethinking Score Distillation as a Bridge Between Image Distributions [97.27476302077545]
  We show that our method seeks to transport corrupted images (source) to the natural image distribution (target)<n>Our method can be easily applied across many domains, matching or beating the performance of specialized methods.<n>We demonstrate its utility in text-to-2D, text-based NeRF optimization, translating paintings to real images, optical illusion generation, and 3D sketch-to-real.
  arXiv  Detail & Related papers  (2024-06-13T17:59:58Z)
• Training on Thin Air: Improve Image Classification with Generated Data [28.96941414724037]
  Diffusion Inversion is a simple yet effective method to generate diverse, high-quality training data for image classification. Our approach captures the original data distribution and ensures data coverage by inverting images to the latent space of Stable Diffusion. We identify three key components that allow our generated images to successfully supplant the original dataset.
  arXiv  Detail & Related papers  (2023-05-24T16:33:02Z)
• High-Fidelity Synthesis with Disentangled Representation [60.19657080953252]
  We propose an Information-Distillation Generative Adrial Network (ID-GAN) for disentanglement learning and high-fidelity synthesis. Our method learns disentangled representation using VAE-based models, and distills the learned representation with an additional nuisance variable to the separate GAN-based generator for high-fidelity synthesis. Despite the simplicity, we show that the proposed method is highly effective, achieving comparable image generation quality to the state-of-the-art methods using the disentangled representation.
  arXiv  Detail & Related papers  (2020-01-13T14:39:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.