DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers

• URL: http://arxiv.org/abs/2605.16732v1
• Date: Sat, 16 May 2026 00:52:00 GMT
• Title: DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
• Abstract summary: Diffusion Transformers (DiTs) achieve state-of-the-art image generation quality but incur substantial memory and computational costs at inference.<n>Existing approaches, including smoothing-based methods, mixed-precision schemes, rotation techniques, and low-rank residual methods, partially mitigate this issue but still leave a noticeable gap to FP16/BF16 performance.<n>We introduce DiRotQ, a W4A4 PTQ framework that mitigates this degradation through rotation-aware activation quantization.
• Score: 3.0583214514538084
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Diffusion Transformers (DiTs) achieve state-of-the-art image generation quality but incur substantial memory and computational costs at inference. While aggressive Post-Training Quantization (PTQ) to 4-bit precision offers significant efficiency gains, it typically results in severe quality degradation. Existing approaches, including smoothing-based methods, mixed-precision schemes, rotation techniques, and low-rank residual methods, partially mitigate this issue but still leave a noticeable gap to FP16/BF16 performance. In this work, we introduce DiRotQ, a W4A4 PTQ framework that mitigates this degradation through rotation-aware activation quantization. DiRotQ identifies a low-rank subspace capturing dominant activation variance via Principal Component Analysis (PCA), preserving coefficients in this subspace at higher precision while quantizing the remaining components to 4-bit. Activations are rotated into the PCA basis at inference time using calibration-derived orthogonal transformations, while the inverse rotation is fused into the layer weights offline. Combined with GPTQ-based weight quantization, DiRotQ achieves an FID (lower is better) of 15.9 and PSNR (higher is better) of 19.1 dB on PixArt-Σ over the MJHQ-30K dataset, outperforming the prior state-of-the-art SVDQuant (FID 18.9, PSNR 17.6) under the same INT W4A4 setting. Beyond standard metrics, we introduce a VLM-as-a-Judge evaluation protocol for diffusion model quantization, the first such evaluation in this setting, providing a more holistic assessment of perceptual quality and prompt alignment under aggressive compression. On the systems side, we implement a Triton-based custom kernel to enable efficient end-to-end inference, reducing memory usage of the 12B FLUX.1-dev model by 2.1x and delivering 2.3x speedup over the BF16 baseline, on a 24 GB RTX 4090 GPU.

Related papers

• HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization [48.87937677659571]
  We introduce HARP (Hadamard-preconditioned Adaptive Rotation Processor), a learnable structured two-sided processor that replaces fixed Hadamard mixing.<n>Across 2-4 bit settings on models ranging from 1B to 70B parameters, HARP improves perplexity and zero-shot accuracy over fixed RHT.<n>HarP preserves deployment efficiency, reaching 128 tok/s versus 61 tok/s for FP16.
  arXiv  Detail & Related papers  (2026-05-28T12:24:15Z)
• TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization [13.984796236174331]
  We propose TORQ (Two-level Orthogonal Rotation for MXFP4 Quantization) as a training-free Post-Training Quantization framework.<n>We show that TORQ significantly improves the accuracy of MXFP4 activation quantization compared to existing methods.
  arXiv  Detail & Related papers  (2026-05-19T09:05:47Z)
• LoRaQ: Optimized Low Rank Approximation for 4-bit Quantization [39.25504627568235]
  Post-training quantization (PTQ) is essential for deploying large diffusion transformers on resource-constrained hardware.<n>LoRaQ (Low-Rank Approximated Quantization) is a simple, data-free calibration approach that optimize quantization error compensation.<n>We demonstrate that LoRaQ outperforms the state-of-the-art methods in their native implementations on Pixart-$$ and SANA.
  arXiv  Detail & Related papers  (2026-04-20T11:37:10Z)
• Q-DiT4SR: Exploration of Detail-Preserving Diffusion Transformer Quantization for Real-World Image Super-Resolution [47.12618295041499]
  We propose Q-DiT4SR, the first PTQ framework specifically tailored for DiT-based Real-ISR.<n>We propose H-SVD, a hierarchical SVD that integrates a global low-rank branch with a local block-wise rank-1 branch under a matched parameter budget.<n> Experiments on multiple real-world datasets demonstrate that our Q-DiT4SR achieves SOTA performance under both W4A6 and W4A4 settings.
  arXiv  Detail & Related papers  (2026-02-01T15:07:59Z)
• Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization [77.67818998672516]
  We present the first comprehensive study of MXFP4 and NVFP4 for post-training quantization.<n>We introduce Micro-Rotated-GPTQ (MR-GPTQ), a variant of the classic GPTQ quantization algorithm.<n>We show that MR-GPTQ matches or outperforms state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2025-09-27T09:22:21Z)
• Quantized Visual Geometry Grounded Transformer [67.15451442018258]
  This paper proposes the first Quantization framework for VGGTs, namely QuantVGGT.<n>We introduce Dual-Smoothed Fine-Grained Quantization, which integrates pre-global Hadamard rotation and post-local channel smoothing.<n>We also design Noise-Filtered Diverse Sampling, which filters outliers via deep-layer statistics.
  arXiv  Detail & Related papers  (2025-09-25T15:17:11Z)
• ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals [10.860081994662645]
  Post-training quantization of large language models (LLMs) holds the promise in reducing the prohibitive computational cost at inference time.<n>We propose ResQ, a PTQ method that pushes further the state-of-the-art.<n>We demonstrate that ResQ outperforms recent uniform and mixed precision PTQ methods on a variety of benchmarks.
  arXiv  Detail & Related papers  (2024-12-18T22:01:55Z)
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models [61.474101404805545]
  Diffusion models can generate high-quality images, but as they scale, rising memory demands and higher latency pose deployment challenges.<n>We propose SVDQuant, a new 4-bit quantization paradigm to overcome this limitation.<n>We reduce the memory usage for the 12B FLUX.1 models by 3.5$times$, achieving 3.0$times$ speedup over the 4-bit weight-only quantization (W4A16) baseline.
  arXiv  Detail & Related papers  (2024-11-07T18:59:58Z)
• 2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution [83.09117439860607]
  Low-bit quantization has become widespread for compressing image super-resolution (SR) models for edge deployment. It is notorious that low-bit quantization degrades the accuracy of SR models compared to their full-precision (FP) counterparts. We present a dual-stage low-bit post-training quantization (PTQ) method for image super-resolution, namely 2DQuant, which achieves efficient and accurate SR under low-bit quantization.
  arXiv  Detail & Related papers  (2024-06-10T06:06:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.