ConvRot: Rotation-Based Plug-and-Play 4-bit Quantization for Diffusion Transformers

• URL: http://arxiv.org/abs/2512.03673v1
• Date: Wed, 03 Dec 2025 11:02:16 GMT
• Title: ConvRot: Rotation-Based Plug-and-Play 4-bit Quantization for Diffusion Transformers
• Abstract summary: As model size increases, the growing memory footprint and inference latency pose significant challenges for practical deployment.<n>Recent studies in large language models (LLMs) show that rotation-based techniques can smooth outliers and enable 4-bit quantization.<n>We propose ConvRot, a group-wise rotation-based quantization method that leverages regular Hadamard transform (RHT) to suppress both row-wise and column-wise outliers.
• Score: 21.65616995056907
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Diffusion transformers have demonstrated strong capabilities in generating high-quality images. However, as model size increases, the growing memory footprint and inference latency pose significant challenges for practical deployment. Recent studies in large language models (LLMs) show that rotation-based techniques can smooth outliers and enable 4-bit quantization, but these approaches often incur substantial overhead and struggle with row-wise outliers in diffusion transformers. To address these challenges, we propose ConvRot, a group-wise rotation-based quantization method that leverages regular Hadamard transform (RHT) to suppress both row-wise and column-wise outliers while reducing complexity from quadratic to linear. Building on this, we design ConvLinear4bit, a plug-and-play module that integrates rotation, quantization, GEMM, and dequantization, enabling W4A4 inference without retraining and preserving visual quality. Experiments on FLUX.1-dev demonstrate a 2.26$\times$ speedup and 4.05$\times$ memory reduction while maintaining image fidelity. To our knowledge, this is the first application of rotation-based quantization for plug-and-play W4A4 inference in diffusion transformers.

Related papers

• KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers [42.11461757540461]
  Post-training quantization of diffusion transformers (DiTs) to W4A4 severely degrades output quality.<n>We propose KroQuant, a PTQ method that applies a learned Kronecker-structured invertible transform to each 32-element block of the activation.
  arXiv  Detail & Related papers  (2026-07-23T15:52:20Z)
• DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers [3.0583214514538084]
  Diffusion Transformers (DiTs) achieve state-of-the-art image generation quality but incur substantial memory and computational costs at inference.<n>Existing approaches, including smoothing-based methods, mixed-precision schemes, rotation techniques, and low-rank residual methods, partially mitigate this issue but still leave a noticeable gap to FP16/BF16 performance.<n>We introduce DiRotQ, a W4A4 PTQ framework that mitigates this degradation through rotation-aware activation quantization.
  arXiv  Detail & Related papers  (2026-05-16T00:52:00Z)
• Q-DiT4SR: Exploration of Detail-Preserving Diffusion Transformer Quantization for Real-World Image Super-Resolution [47.12618295041499]
  We propose Q-DiT4SR, the first PTQ framework specifically tailored for DiT-based Real-ISR.<n>We propose H-SVD, a hierarchical SVD that integrates a global low-rank branch with a local block-wise rank-1 branch under a matched parameter budget.<n> Experiments on multiple real-world datasets demonstrate that our Q-DiT4SR achieves SOTA performance under both W4A6 and W4A4 settings.
  arXiv  Detail & Related papers  (2026-02-01T15:07:59Z)
• Alias-free 4D Gaussian Splatting [27.259991918653427]
  dynamic scene reconstruction methods based on Gaussian Splatting enable real-time rendering and generate realistic images.<n>We propose a maximum sampling frequency formulation for 4D Gaussian Splatting and introduce a 4D scale-adaptive filter and scale loss.<n>Our approach eliminates high-frequency artifacts under increased rendering frequencies while effectively reducing redundant Gaussians in multi-view video reconstruction.
  arXiv  Detail & Related papers  (2025-11-23T09:26:01Z)
• Optimized Minimal 4D Gaussian Splatting [50.29519144753797]
  4D Gaussian Splatting has emerged as a new paradigm for dynamic scene representation, enabling real-time rendering of scenes with complex motions.<n>It faces a major challenge of storage overhead, as millions of Gaussians are required for high-fidelity reconstruction.<n>We present OMG4, a framework that constructs a compact set of salient Gaussians capable of faithfully representing 4D Gaussian models.
  arXiv  Detail & Related papers  (2025-10-04T16:11:13Z)
• Quantized Visual Geometry Grounded Transformer [67.15451442018258]
  This paper proposes the first Quantization framework for VGGTs, namely QuantVGGT.<n>We introduce Dual-Smoothed Fine-Grained Quantization, which integrates pre-global Hadamard rotation and post-local channel smoothing.<n>We also design Noise-Filtered Diverse Sampling, which filters outliers via deep-layer statistics.
  arXiv  Detail & Related papers  (2025-09-25T15:17:11Z)
• S$^2$Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillation [55.35880044416441]
  We propose S$2$Q-VDiT, a post-training quantization framework for video diffusion models (V-DMs)<n>Under W4A6 quantization, S$2$Q-VDiT achieves lossless performance while delivering $3.9times$ model compression and $1.3times$ inference acceleration.
  arXiv  Detail & Related papers  (2025-08-06T02:12:29Z)
• Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping [36.012101195384744]
  Diffusion Transformer (DiT) has now become the preferred choice for building image generation models.<n>DiT is purely composed of a stack of transformer blocks, which renders DiT excellent in scalability like large language models.<n>We propose a post-training quantization framework tailored for Diffusion Transforms to tackle these challenges.
  arXiv  Detail & Related papers  (2025-03-10T05:21:04Z)
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models [61.474101404805545]
  Diffusion models can generate high-quality images, but as they scale, rising memory demands and higher latency pose deployment challenges.<n>We propose SVDQuant, a new 4-bit quantization paradigm to overcome this limitation.<n>We reduce the memory usage for the 12B FLUX.1 models by 3.5$times$, achieving 3.0$times$ speedup over the 4-bit weight-only quantization (W4A16) baseline.
  arXiv  Detail & Related papers  (2024-11-07T18:59:58Z)
• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation [23.99995355561429]
  Post-Training Quantization (PTQ) is an effective method for reducing memory costs and computational complexity.<n>We introduce ViDiT-Q (Video & Image Diffusion Transformer Quantization), tailored specifically for DiT models.<n>We validate the effectiveness of ViDiT-Q across a variety of text-to-image and video models, achieving W8A8 and W4A8 with negligible degradation in visual quality and metrics.
  arXiv  Detail & Related papers  (2024-06-04T17:57:10Z)
• PTQ4DiT: Post-training Quantization for Diffusion Transformers [52.902071948957186]
  Post-training Quantization (PTQ) has emerged as a fast and data-efficient solution that can significantly reduce computation and memory footprint. We propose PTQ4DiT, a specifically designed PTQ method for DiTs. We demonstrate that our PTQ4DiT successfully quantizes DiTs to 8-bit precision while preserving comparable generation ability.
  arXiv  Detail & Related papers  (2024-05-25T02:02:08Z)
• Orthonormal Convolutions for the Rotation Based Iterative Gaussianization [64.44661342486434]
  This paper elaborates an extension of rotation-based iterative Gaussianization, RBIG, which makes image Gaussianization possible. In images its application has been restricted to small image patches or isolated pixels, because rotation in RBIG is based on principal or independent component analysis. We present the emphConvolutional RBIG: an extension that alleviates this issue by imposing that the rotation in RBIG is a convolution.
  arXiv  Detail & Related papers  (2022-06-08T12:56:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.