DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization

• URL: http://arxiv.org/abs/2604.17789v2
• Date: Tue, 21 Apr 2026 07:37:48 GMT
• Title: DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization
• Abstract summary: We propose DuQuant++, which adapts the outlier-aware fine-grained rotation of DuQuant to the MXFP4 format.<n>Experiments on the LLaMA-3 family under MXFP4 W4A4 quantization show that DuQuant++ consistently achieves state-of-the-art performance.
• Score: 47.19478866645546
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The MXFP4 microscaling format, which partitions tensors into blocks of 32 elements sharing an E8M0 scaling factor, has emerged as a promising substrate for efficient LLM inference, backed by native hardware support on NVIDIA Blackwell Tensor Cores. However, activation outliers pose a unique challenge under this format: a single outlier inflates the shared block scale, compressing the effective dynamic range of the remaining elements and causing significant quantization error. Existing rotation-based remedies, including randomized Hadamard and learnable rotations, are data-agnostic and therefore unable to specifically target the channels where outliers concentrate. We propose DuQuant++, which adapts the outlier-aware fine-grained rotation of DuQuant to the MXFP4 format by aligning the rotation block size with the microscaling group size (B{=}32). Because each MXFP4 group possesses an independent scaling factor, the cross-block variance issue that necessitates dual rotations and a zigzag permutation in the original DuQuant becomes irrelevant, enabling DuQuant++ to replace the entire pipeline with a single outlier-aware rotation, which halves the online rotation cost while simultaneously smoothing the weight distribution. Extensive experiments on the LLaMA-3 family under MXFP4 W4A4 quantization show that DuQuant++ consistently achieves state-of-the-art performance. Our code is available at https://github.com/Hsu1023/DuQuant-v2.

Related papers

• SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference [4.219937026460372]
  SharQ bridges activation sparsity and FP4 quantization through an online sparse--dense decomposition.<n>A sparse FP4 GEMM processes the backbone while a dense FP4 GEMM compensates for both mask-induced activation loss and sparse-path quantization error.
  arXiv  Detail & Related papers  (2026-06-25T04:19:04Z)
• TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization [13.984796236174331]
  We propose TORQ (Two-level Orthogonal Rotation for MXFP4 Quantization) as a training-free Post-Training Quantization framework.<n>We show that TORQ significantly improves the accuracy of MXFP4 activation quantization compared to existing methods.
  arXiv  Detail & Related papers  (2026-05-19T09:05:47Z)
• Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization [77.67818998672516]
  We present the first comprehensive study of MXFP4 and NVFP4 for post-training quantization.<n>We introduce Micro-Rotated-GPTQ (MR-GPTQ), a variant of the classic GPTQ quantization algorithm.<n>We show that MR-GPTQ matches or outperforms state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2025-09-27T09:22:21Z)
• Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizer [7.833559115428751]
  Rotate, Clip, and Partition (RCP) is a quantization-aware training (QAT) approach.<n>RCP integrates recent rotation techniques with a novel non-uniform weight quantizer design.<n>RCP can compress LLaMA-2-7B to W2A4KV4 with a loss of only 2.84 WikiText2 ppl and 5.29 times reduced memory footprint.
  arXiv  Detail & Related papers  (2025-02-17T08:12:34Z)
• AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference [6.699442219974261]
  AMXFP4 is a 4-bit asymmetric FP format that handles both issues using asymmetric shared scales.<n>AMXFP4 outperforms MXFP4 by 3% on VQA and exceeds rotation-based methods by 1.6% on CSQA.
  arXiv  Detail & Related papers  (2024-11-15T03:11:19Z)
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models [61.474101404805545]
  Diffusion models can generate high-quality images, but as they scale, rising memory demands and higher latency pose deployment challenges.<n>We propose SVDQuant, a new 4-bit quantization paradigm to overcome this limitation.<n>We reduce the memory usage for the 12B FLUX.1 models by 3.5$times$, achieving 3.0$times$ speedup over the 4-bit weight-only quantization (W4A16) baseline.
  arXiv  Detail & Related papers  (2024-11-07T18:59:58Z)
• FlatQuant: Flatness Matters for LLM Quantization [58.28221892035609]
  We propose FlatQuant, a new post-training quantization approach that enhances the flatness of weights and activations.<n>Our approach identifies optimal affine transformations for each linear layer, calibrated in hours via a lightweight objective.<n>It achieves less than 1% accuracy drop for W4A4 quantization on the LLaMA-3-70B model, surpassing SpinQuant by 7.5%.
  arXiv  Detail & Related papers  (2024-10-12T08:10:28Z)
• SpinQuant: LLM quantization with learned rotations [49.07335692298487]
  Post-training quantization (PTQ) techniques applied to weights, activations, and the KV cache greatly reduce memory usage, latency, and power consumption of Large Language Models (LLMs)<n>We identify a collection of applicable rotation parameterizations that lead to identical outputs in full-precision Transformer architectures while enhancing quantization accuracy.<n>We propose SpinQuant, a novel approach that incorporates learned rotation matrices for optimal quantized network accuracy.
  arXiv  Detail & Related papers  (2024-05-26T02:15:49Z)
• QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs [72.26197676852958]
  We introduce QuaRot, a new Quantization scheme based on Rotations. QuaRot quantizes end-to-end, including all weights, activations, and KV cache in 4 bits. Our 4-bit quantized LLaMa2-70B model has losses of at most 0.47 WikiText-2 perplexity and retains 99% of the zero-shot performance.
  arXiv  Detail & Related papers  (2024-03-30T19:20:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.