Learning Quantized Adaptive Conditions for Diffusion Models

• URL: http://arxiv.org/abs/2409.17487v1
• Date: Thu, 26 Sep 2024 02:49:51 GMT
• Title: Learning Quantized Adaptive Conditions for Diffusion Models
• Authors: Yuchen Liang, Yuchuan Tian, Lei Yu, Huao Tang, Jie Hu, Xiangzhong Fang, Hanting Chen,
• Abstract summary: We propose a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions. Our method incurs only an additional 1% of training parameters, eliminates the need for extra regularization terms, yet significantly better sample quality.
• Score: 19.9601581920218
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The curvature of ODE trajectories in diffusion models hinders their ability to generate high-quality images in a few number of function evaluations (NFE). In this paper, we propose a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions. By employing a extremely light-weight quantized encoder, our method incurs only an additional 1% of training parameters, eliminates the need for extra regularization terms, yet achieves significantly better sample quality. Our approach accelerates ODE sampling while preserving the downstream task image editing capabilities of SDE techniques. Extensive experiments verify that our method can generate high quality results under extremely limited sampling costs. With only 6 NFE, we achieve 5.14 FID on CIFAR-10, 6.91 FID on FFHQ 64x64 and 3.10 FID on AFHQv2.

Related papers

• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step. To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration. Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• PQD: Post-training Quantization for Efficient Diffusion Models [4.809939957401427]
  We propose a novel post-training quantization for diffusion models (PQD) We show that our proposed method is able to directly quantize full-precision diffusion models into 8-bit or 4-bit models while maintaining comparable performance in a training-free manner.
  arXiv  Detail & Related papers  (2024-12-30T19:55:59Z)
• Simple and Fast Distillation of Diffusion Models [39.79747569096888]
  We propose Simple and Fast Distillation (SFD) of diffusion models, which simplifies the paradigm used in existing methods. SFD achieves 4.53 FID (NFE=2) on CIFAR-10 with only 0.64 hours of fine-tuning on a single NVIDIA A100 GPU.
  arXiv  Detail & Related papers  (2024-09-29T12:13:06Z)
• Unveiling Advanced Frequency Disentanglement Paradigm for Low-Light Image Enhancement [61.22119364400268]
  We propose a novel low-frequency consistency method, facilitating improved frequency disentanglement optimization. Noteworthy improvements are showcased across five popular benchmarks, with up to 7.68dB gains on PSNR achieved for six state-of-the-art models. Our approach maintains efficiency with only 88K extra parameters, setting a new standard in the challenging realm of low-light image enhancement.
  arXiv  Detail & Related papers  (2024-09-03T06:19:03Z)
• Diffusion Models Are Innate One-Step Generators [2.3359837623080613]
  Diffusion Models (DMs) can generate remarkable high-quality results. DMs' layers are differentially activated at different time steps, leading to an inherent capability to generate images in a single step. Our method achieves the SOTA results on CIFAR-10, AFHQv2 64x64 (FID 1.23), FFHQ 64x64 (FID 0.85) and ImageNet 64x64 (FID 1.16) with great efficiency.
  arXiv  Detail & Related papers  (2024-05-31T11:14:12Z)
• QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning [52.157939524815866]
  In this paper, we empirically unravel three properties in quantized diffusion models that compromise the efficacy of current methods. We identify two critical types of quantized layers: those holding vital temporal information and those sensitive to reduced bit-width. Our method is evaluated over three high-resolution image generation tasks and achieves state-of-the-art performance under various bit-width settings.
  arXiv  Detail & Related papers  (2024-02-06T03:39:44Z)
• Fast ODE-based Sampling for Diffusion Models in Around 5 Steps [17.500594480727617]
  We propose Approximate MEan-Direction solver (AMED-r) that eliminates truncation errors by directly learning the mean direction for fast sampling. Our method can be easily used as a plugin to further improve existing ODE-based samplers.
  arXiv  Detail & Related papers  (2023-11-30T13:07:19Z)
• Manifold Preserving Guided Diffusion [121.97907811212123]
  Conditional image generation still faces challenges of cost, generalizability, and the need for task-specific training. We propose Manifold Preserving Guided Diffusion (MPGD), a training-free conditional generation framework.
  arXiv  Detail & Related papers  (2023-11-28T02:08:06Z)
• ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution [84.73658185158222]
  We propose a diffusion model-based super-resolution method called ACDMSR. Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
  arXiv  Detail & Related papers  (2023-07-03T06:49:04Z)
• Low-Light Image Enhancement with Wavelet-based Diffusion Models [50.632343822790006]
  Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. We propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL.
  arXiv  Detail & Related papers  (2023-06-01T03:08:28Z)
• On Distillation of Guided Diffusion Models [94.95228078141626]
  We propose an approach to distilling classifier-free guided diffusion models into models that are fast to sample from. For standard diffusion models trained on the pixelspace, our approach is able to generate images visually comparable to that of the original model. For diffusion models trained on the latent-space (e.g., Stable Diffusion), our approach is able to generate high-fidelity images using as few as 1 to 4 denoising steps.
  arXiv  Detail & Related papers  (2022-10-06T18:03:56Z)
• Improved Vector Quantized Diffusion Models [34.23016989464389]
  VQ-Diffusion is a powerful generative model for text-to-image synthesis. It can generate low-quality samples or weakly correlated images with text input. We propose two techniques to further improve the sample quality of VQ-Diffusion.
  arXiv  Detail & Related papers  (2022-05-31T17:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.