ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization

• URL: http://arxiv.org/abs/2606.07618v1
• Date: Sat, 30 May 2026 08:26:16 GMT
• Title: ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization
• Abstract summary: We propose ScaleSweep as a simple and efficient scale optimization method for NVFP4 quantization.<n> Experiments on Llama and Qwen models demonstrate that ScaleSweep consistently improves quantization performance.<n>In particular, under aggressive end-to-end quantization of weights, activations, KV cache, and query states, ScaleSweep preserves more than 93% of the full-precision performance.
• Score: 40.425536766574844
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: NVFP4 is a recently introduced hardware-supported FP4 format that improves the fidelity of 4-bit quantization through fine-grained block scales. However, existing NVFP4 scale initialization methods still primarily rely on AbsMax initialization, which leaves a noticeable gap to the optimal solution. To address this, we propose ScaleSweep, a simple and efficient scale optimization method that sweeps over feasible block scale candidates and selects the candidate that minimizes a target objective. We further provide a theoretical analysis of NVFP4 quantization and derive both lower and upper bounds for the required sweep range under mean square error (MSE) and weighted mean square error (WMSE) between the original tensor and the quantized reconstructed tensor. The proposed bounds substantially reduce the sweep space while preserving the optimal candidate, enabling negligible overhead compared with the baseline quantization operators. Experiments on Llama and Qwen models demonstrate that ScaleSweep consistently improves quantization performance over existing initialization methods and further narrows the gap to full precision. In particular, under aggressive end-to-end quantization of weights, activations, KV cache, and query states, ScaleSweep preserves more than 93% of the full-precision performance.

Related papers

• H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference [27.189669587707858]
  H-Scale is a lightweight post-processing method for NVFP4 per-group scale refinement.<n>It is designed as a drop-in replacement for RTN-style scale selection in diverse NVFP4 pipelines.<n> experiments show that H-Scale generally improves a broad range of NVFP4 baselines.
  arXiv  Detail & Related papers  (2026-08-28T09:22:17Z)
• FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling [23.78963716646984]
  FOCUS is a post-training quantization framework with end-to-end scale learning for FP4 optimization.<n>It relaxes the tight coupling between quantization and dequantization scales with a learnable full-precision coefficient.<n>Experiments show that FOCUS achieves state-of-the-art FP4 accuracy under both MXFP4 and NVFP4 formats.
  arXiv  Detail & Related papers  (2026-08-03T07:54:56Z)
• Stable FP4 Training via Transposition-Invariant Block Quantization [14.373963304887754]
  In conventional 1D block quantization, forward and backward passes assign di erent scaling factors to the same values after transposition.<n>We propose a low-precision training framework based on 2D block FP4 quantization, which enforces transposition-invariant scaling and preserves consistency between forward and backward computations.<n>Our approach achieves stable end-to-end FP4 training and closely matches BF16 performance, with less than 1.3% degradation in perplexity and downstream accuracy.
  arXiv  Detail & Related papers  (2026-07-27T18:03:34Z)
• SOAR: Scale Optimization for Accurate Reconstruction in NVFP4 Quantization [19.022444007775896]
  NVFP4 has recently emerged as an efficient 4-bit microscaling format for large language models.<n>Existing methods often yield suboptimal performance due to inflexible scale selection and the coupled treatment of quantization and dequantization scales.<n>We propose Scale Optimization for Accurate Reconstruction (SOAR), a novel post-training quantization framework that improves the accuracy of NVFP4 quantization.
  arXiv  Detail & Related papers  (2026-05-12T15:13:18Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• Fine-tuning Quantized Neural Networks with Zeroth-order Optimization [21.0540879091664]
  We propose Quantized Zeroth-order Optimization (QZO), a simple yet effective approach that perturbs the continuous quantization scale for gradient estimation.<n>QZO can reduce the total memory cost by more than 18$times$ for 4-bit LLMs, and enables fine-tuning Llama-2-13B within a single 24GB GPU.
  arXiv  Detail & Related papers  (2025-05-19T17:55:15Z)
• Improving Block-Wise LLM Quantization by 4-bit Block-Wise Optimal Float (BOF4): Analysis and Variations [22.127873567034825]
  Large language models (LLMs) demand extensive memory capacity during both fine-tuning and inference.<n>Existing methods apply block-wise quantization techniques, such as NF4 and AF4, to the network weights.<n>We show that these quantization techniques incur suboptimal quantization errors.
  arXiv  Detail & Related papers  (2025-05-10T14:00:15Z)
• Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics [64.62231094774211]
  Statefuls (e.g., Adam) maintain auxiliary information even 2x the model size in order to achieve optimal convergence.<n>SOLO enables Adam-styles to maintain quantized states with precision as low as 3 bits, or even 2 bits.<n>SOLO can thus be seamlessly applied to Adam-styles, leading to substantial memory savings with minimal accuracy loss.
  arXiv  Detail & Related papers  (2025-05-01T06:47:45Z)
• GAQAT: gradient-adaptive quantization-aware training for domain generalization [54.31450550793485]
  We propose a novel Gradient-Adaptive Quantization-Aware Training (GAQAT) framework for DG.<n>Our approach begins by identifying the scale-gradient conflict problem in low-precision quantization.<n>Extensive experiments validate the effectiveness of the proposed GAQAT framework.
  arXiv  Detail & Related papers  (2024-12-07T06:07:21Z)
• FlatQuant: Flatness Matters for LLM Quantization [58.28221892035609]
  We propose FlatQuant, a new post-training quantization approach that enhances the flatness of weights and activations.<n>Our approach identifies optimal affine transformations for each linear layer, calibrated in hours via a lightweight objective.<n>It achieves less than 1% accuracy drop for W4A4 quantization on the LLaMA-3-70B model, surpassing SpinQuant by 7.5%.
  arXiv  Detail & Related papers  (2024-10-12T08:10:28Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [63.118592279833656]
  Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs)<n>We propose SliM-LLM, a salience-driven mixed-precision quantization framework that allocates bit-widths at the group-wise.<n> Experiments show that SliM-LLM achieves superior performance across various LLMs at low bit-widths.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• AffineQuant: Affine Transformation Quantization for Large Language Models [58.45460102764]
  Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for Large-scale Language Models (LLMs) limit the optimization scope to scaling transformations between pre- and post-quantization weights. In this paper, we advocate for the direct optimization using equivalent Affine transformations in PTQ (AffineQuant)
  arXiv  Detail & Related papers  (2024-03-19T08:40:21Z)
• LLM-FP4: 4-Bit Floating-Point Quantized Transformers [38.23587031169402]
  We propose LLM-FP4 for quantizing both weights and activations in large language models (LLMs) down to 4-bit floating-point values. Compared to integer quantization, floating-point (FP) quantization is more flexible and can better handle long-tail or bell-shaped distributions. Our method, for the first time, can quantize both weights and activations in the LLaMA-13B to only 4-bit and achieves an average score of 63.1.
  arXiv  Detail & Related papers  (2023-10-25T17:59:32Z)
• LG-LSQ: Learned Gradient Linear Symmetric Quantization [3.6816597150770387]
  Deep neural networks with lower precision weights have advantages in terms of the cost of memory space and accelerator power. The main challenge associated with the quantization algorithm is maintaining accuracy at low bit-widths. We propose learned gradient linear symmetric quantization (LG-LSQ) as a method for quantizing weights and activation functions to low bit-widths.
  arXiv  Detail & Related papers  (2022-02-18T03:38:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.