Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance

• URL: http://arxiv.org/abs/2409.01347v1
• Date: Mon, 2 Sep 2024 16:01:38 GMT
• Title: Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance
• Authors: Cunzheng Wang, Ziyuan Guo, Yuxuan Duan, Huaxia Li, Nemo Chen, Xu Tang, Yao Hu,
• Abstract summary: We introduce Target-Driven Distillation (TDD) to accelerate generative tasks of diffusion models. TDD adopts delicate selection strategy of target timesteps, increasing the training efficiency. It can be equipped with non-equidistant sampling and x0 clipping, enabling a more flexible and accurate way for image sampling.
• Score: 17.826285840875556
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Consistency distillation methods have demonstrated significant success in accelerating generative tasks of diffusion models. However, since previous consistency distillation methods use simple and straightforward strategies in selecting target timesteps, they usually struggle with blurs and detail losses in generated images. To address these limitations, we introduce Target-Driven Distillation (TDD), which (1) adopts a delicate selection strategy of target timesteps, increasing the training efficiency; (2) utilizes decoupled guidances during training, making TDD open to post-tuning on guidance scale during inference periods; (3) can be optionally equipped with non-equidistant sampling and x0 clipping, enabling a more flexible and accurate way for image sampling. Experiments verify that TDD achieves state-of-the-art performance in few-step generation, offering a better choice among consistency distillation models.

Related papers

• Diffusion Distillation With Direct Preference Optimization For Efficient 3D LiDAR Scene Completion [25.55163699029964]
  This paper proposes Distillation-DPO, a novel diffusion distillation framework for LiDAR scene completion with preference aligment. Our method is the first to explore adopting preference learning in distillation to the best of our knowledge and provide insights into preference-aligned distillation.
  arXiv  Detail & Related papers  (2025-04-15T17:57:13Z)
• TraFlow: Trajectory Distillation on Pre-Trained Rectified Flow [31.56008127287467]
  We propose a trajectory distillation method, modelname, that enjoys the benefits of both and enables few-step generations. TraFlow adopts the settings of consistency trajectory models, and further enforces the properties of self-consistency and straightness throughout the entire trajectory.
  arXiv  Detail & Related papers  (2025-02-24T08:57:19Z)
• Inference-Time Diffusion Model Distillation [59.350789627086456]
  We introduce Distillation++, a novel inference-time distillation framework. Inspired by recent advances in conditional sampling, our approach recasts student model sampling as a proximal optimization problem. We integrate distillation optimization during reverse sampling, which can be viewed as teacher guidance.
  arXiv  Detail & Related papers  (2024-12-12T02:07:17Z)
• Relational Diffusion Distillation for Efficient Image Generation [27.127061578093674]
  Diffusion model's high delay hinders its wide application in edge devices with scarce computing resources. We propose Diffusion Distillation (RDD), a novel distillation method tailored specifically for distilling diffusion models. Our proposed RDD leads to 1.47 FID decrease under 1 sampling step compared to state-of-the-art diffusion distillation methods and achieving 256x speed-up.
  arXiv  Detail & Related papers  (2024-10-10T07:40:51Z)
• Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization [97.35427957922714]
  We present an algorithm named pairwise sample optimization (PSO), which enables the direct fine-tuning of an arbitrary timestep-distilled diffusion model. PSO introduces additional reference images sampled from the current time-step distilled model, and increases the relative likelihood margin between the training images and reference images. We show that PSO can directly adapt distilled models to human-preferred generation with both offline and online-generated pairwise preference image data.
  arXiv  Detail & Related papers  (2024-10-04T07:05:16Z)
• One Step Diffusion-based Super-Resolution with Time-Aware Distillation [60.262651082672235]
  Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. Recent techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. We propose a time-aware diffusion distillation method, named TAD-SR, to accomplish effective and efficient image super-resolution.
  arXiv  Detail & Related papers  (2024-08-14T11:47:22Z)
• Guided Score identity Distillation for Data-Free One-Step Text-to-Image Generation [62.30570286073223]
  Diffusion-based text-to-image generation models have demonstrated the ability to produce images aligned with textual descriptions. We introduce a data-free guided distillation method that enables the efficient distillation of pretrained Diffusion models without access to the real training data. By exclusively training with synthetic images generated by its one-step generator, our data-free distillation method rapidly improves FID and CLIP scores, achieving state-of-the-art FID performance while maintaining a competitive CLIP score.
  arXiv  Detail & Related papers  (2024-06-03T17:44:11Z)
• EM Distillation for One-step Diffusion Models [65.57766773137068]
  We propose a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of quality. We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process.
  arXiv  Detail & Related papers  (2024-05-27T05:55:22Z)
• Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation [61.03530321578825]
  We introduce Score identity Distillation (SiD), an innovative data-free method that distills the generative capabilities of pretrained diffusion models into a single-step generator. SiD not only facilitates an exponentially fast reduction in Fr'echet inception distance (FID) during distillation but also approaches or even exceeds the FID performance of the original teacher diffusion models.
  arXiv  Detail & Related papers  (2024-04-05T12:30:19Z)
• AddSR: Accelerating Diffusion-based Blind Super-Resolution with Adversarial Diffusion Distillation [43.62480338471837]
  Blind super-resolution methods based on stable diffusion showcase formidable generative capabilities in reconstructing clear high-resolution images with intricate details from low-resolution inputs. Their practical applicability is often hampered by poor efficiency, stemming from the requirement of thousands or hundreds of sampling steps. Inspired by the efficient adversarial diffusion distillation (ADD), we designnameto address this issue by incorporating the ideas of both distillation and ControlNet.
  arXiv  Detail & Related papers  (2024-04-02T08:07:38Z)
• Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation [24.236841051249243]
  Distillation methods aim to shift the model from many-shot to single-step inference. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:51:43Z)
• Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping [75.72212215739746]
  Trajectory Consistency Distillation (TCD) encompasses trajectory consistency function and strategic sampling. TCD significantly enhances image quality at low NFEs but also yields more detailed results compared to the teacher model.
  arXiv  Detail & Related papers  (2024-02-29T13:44:14Z)
• BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping [64.54271680071373]
  Diffusion models have demonstrated excellent potential for generating diverse images. Knowledge distillation has been recently proposed as a remedy that can reduce the number of inference steps to one or a few. We present a novel technique called BOOT, that overcomes limitations with an efficient data-free distillation algorithm.
  arXiv  Detail & Related papers  (2023-06-08T20:30:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.