Mean-Shift Distillation for Diffusion Mode Seeking

• URL: http://arxiv.org/abs/2502.15989v1
• Date: Fri, 21 Feb 2025 22:58:56 GMT
• Title: Mean-Shift Distillation for Diffusion Mode Seeking
• Authors: Vikas Thamizharasan, Nikitas Chatzis, Iliyan Georgiev, Matthew Fisher, Difan Liu, Nanxuan Zhao, Evangelos Kalogerakis, Michal Lukac,
• Abstract summary: Mean-shift distillation provides a proxy for the gradient of the diffusion output distribution.<n>We show that it exhibits superior mode alignment as well as improved convergence in both synthetic and practical setups.
• Score: 29.341453176830225
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present mean-shift distillation, a novel diffusion distillation technique that provides a provably good proxy for the gradient of the diffusion output distribution. This is derived directly from mean-shift mode seeking on the distribution, and we show that its extrema are aligned with the modes. We further derive an efficient product distribution sampling procedure to evaluate the gradient. Our method is formulated as a drop-in replacement for score distillation sampling (SDS), requiring neither model retraining nor extensive modification of the sampling procedure. We show that it exhibits superior mode alignment as well as improved convergence in both synthetic and practical setups, yielding higher-fidelity results when applied to both text-to-image and text-to-3D applications with Stable Diffusion.

Related papers

• Inference-Time Diffusion Model Distillation [59.350789627086456]
  We introduce Distillation++, a novel inference-time distillation framework. Inspired by recent advances in conditional sampling, our approach recasts student model sampling as a proximal optimization problem. We integrate distillation optimization during reverse sampling, which can be viewed as teacher guidance.
  arXiv  Detail & Related papers  (2024-12-12T02:07:17Z)
• DDIL: Improved Diffusion Distillation With Imitation Learning [57.3467234269487]
  Diffusion models excel at generative modeling (e.g., text-to-image) but sampling requires multiple denoising network passes. Progressive distillation or consistency distillation have shown promise by reducing the number of passes. We show that DDIL consistency improves on baseline algorithms of progressive distillation (PD), Latent consistency models (LCM) and Distribution Matching Distillation (DMD2)
  arXiv  Detail & Related papers  (2024-10-15T18:21:47Z)
• Relational Diffusion Distillation for Efficient Image Generation [27.127061578093674]
  Diffusion model's high delay hinders its wide application in edge devices with scarce computing resources. We propose Diffusion Distillation (RDD), a novel distillation method tailored specifically for distilling diffusion models. Our proposed RDD leads to 1.47 FID decrease under 1 sampling step compared to state-of-the-art diffusion distillation methods and achieving 256x speed-up.
  arXiv  Detail & Related papers  (2024-10-10T07:40:51Z)
• EM Distillation for One-step Diffusion Models [65.57766773137068]
  We propose a maximum likelihood-based approach that distills a diffusion model to a one-step generator model with minimal loss of quality.<n>We develop a reparametrized sampling scheme and a noise cancellation technique that together stabilizes the distillation process.
  arXiv  Detail & Related papers  (2024-05-27T05:55:22Z)
• Distilling Diffusion Models into Conditional GANs [90.76040478677609]
  We distill a complex multistep diffusion model into a single-step conditional GAN student model. For efficient regression loss, we propose E-LatentLPIPS, a perceptual loss operating directly in diffusion model's latent space. We demonstrate that our one-step generator outperforms cutting-edge one-step diffusion distillation models.
  arXiv  Detail & Related papers  (2024-05-09T17:59:40Z)
• Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation [24.236841051249243]
  Distillation methods aim to shift the model from many-shot to single-step inference. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models.
  arXiv  Detail & Related papers  (2024-03-18T17:51:43Z)
• ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting [70.83632337581034]
  Diffusion-based image super-resolution (SR) methods are mainly limited by the low inference speed. We propose a novel and efficient diffusion model for SR that significantly reduces the number of diffusion steps. Our method constructs a Markov chain that transfers between the high-resolution image and the low-resolution image by shifting the residual.
  arXiv  Detail & Related papers  (2023-07-23T15:10:02Z)
• Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution [82.50210340928173]
  randomness of diffusion models results in ineffectiveness and instability, making it challenging for users to guarantee the quality of SR results. We propose a plug-and-play sampling method that owns the potential to benefit a series of diffusion-based SR methods. The quality of SR results sampled by the proposed method with fewer steps outperforms the quality of results sampled by current methods with randomness from the same pre-trained diffusion-based SR model.
  arXiv  Detail & Related papers  (2023-05-24T17:09:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.