Related papers: Text Embedding Knows How to Quantize Text-Guided Diffusion Models

Text Embedding Knows How to Quantize Text-Guided Diffusion Models

URL: http://arxiv.org/abs/2507.10340v3
Date: Mon, 04 Aug 2025 04:07:02 GMT
Title: Text Embedding Knows How to Quantize Text-Guided Diffusion Models
Authors: Hongjae Lee, Myungjun Son, Dongjea Kang, Seung-Won Jung,
Abstract summary: We propose a novel quantization method dubbed Quantization of Language-to-Image diffusion models using text Prompts (QLIP)<n>QLIP leverages text prompts to guide the selection of bit precision for every layer at each time step.<n>Our experiments demonstrate the effectiveness of QLIP in reducing computational complexity and improving the quality of the generated images.
Score: 9.345515987536244
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Despite the success of diffusion models in image generation tasks such as text-to-image, the enormous computational complexity of diffusion models limits their use in resource-constrained environments. To address this, network quantization has emerged as a promising solution for designing efficient diffusion models. However, existing diffusion model quantization methods do not consider input conditions, such as text prompts, as an essential source of information for quantization. In this paper, we propose a novel quantization method dubbed Quantization of Language-to-Image diffusion models using text Prompts (QLIP). QLIP leverages text prompts to guide the selection of bit precision for every layer at each time step. In addition, QLIP can be seamlessly integrated into existing quantization methods to enhance quantization efficiency. Our extensive experiments demonstrate the effectiveness of QLIP in reducing computational complexity and improving the quality of the generated images across various datasets.

Related papers

DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Models [12.875837358532422]
We analyze the challenges associated with quantizing text-to-image diffusion models from a distributional perspective.<n>We propose Distribution-aware Group Quantization (DGQ), a method that adaptively handles pixel-wise and channel-wise outliers to preserve image quality.<n>Our method demonstrates remarkable performance on datasets such as MS-COCO and PartiPrompts.
arXiv Detail & Related papers (2025-01-08T06:30:31Z)
PQD: Post-training Quantization for Efficient Diffusion Models [4.809939957401427]
We propose a novel post-training quantization for diffusion models (PQD)<n>We show that our proposed method is able to directly quantize full-precision diffusion models into 8-bit or 4-bit models while maintaining comparable performance in a training-free manner.
arXiv Detail & Related papers (2024-12-30T19:55:59Z)
Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss. Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
arXiv Detail & Related papers (2024-07-10T19:58:04Z)
QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning [52.157939524815866]
In this paper, we identify imbalanced activation distributions as a primary source of quantization difficulty.<n>We propose to adjust these distributions through weight finetuning to be more quantization-friendly.<n>Our method demonstrates its efficacy across three high-resolution image generation tasks.
arXiv Detail & Related papers (2024-02-06T03:39:44Z)
Post-training Quantization for Text-to-Image Diffusion Models with Progressive Calibration and Activation Relaxing [49.800746112114375]
We propose a novel post-training quantization method (Progressive and Relaxing) for text-to-image diffusion models. We are the first to achieve quantization for Stable Diffusion XL while maintaining the performance.
arXiv Detail & Related papers (2023-11-10T09:10:09Z)
Effective Quantization for Diffusion Models on CPUs [5.419875683826296]
Quantization, a technique employed to compress deep learning models for enhanced efficiency, presents challenges when applied to diffusion models. We introduce a novel approach to quantize the diffusion models by leveraging both quantization-aware training and distillation. Our results show the quantized models can maintain the high image quality while demonstrating the inference efficiency on CPUs.
arXiv Detail & Related papers (2023-11-02T13:14:01Z)
Towards Accurate Post-training Quantization for Diffusion Models [73.19871905102545]
We propose an accurate data-free post-training quantization framework of diffusion models (ADP-DM) for efficient image generation. Our method outperforms the state-of-the-art post-training quantization of diffusion model by a sizable margin with similar computational cost.
arXiv Detail & Related papers (2023-05-30T04:00:35Z)
Q-Diffusion: Quantizing Diffusion Models [52.978047249670276]
Post-training quantization (PTQ) is considered a go-to compression method for other tasks. We propose a novel PTQ method specifically tailored towards the unique multi-timestep pipeline and model architecture. We show that our proposed method is able to quantize full-precision unconditional diffusion models into 4-bit while maintaining comparable performance.
arXiv Detail & Related papers (2023-02-08T19:38:59Z)
Semantic Image Synthesis via Diffusion Models [174.24523061460704]
Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks.<n>Recent work on semantic image synthesis mainly follows the de facto GAN-based approaches.<n>We propose a novel framework based on DDPM for semantic image synthesis.
arXiv Detail & Related papers (2022-06-30T18:31:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.