Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor

• URL: http://arxiv.org/abs/2605.20402v2
• Date: Fri, 22 May 2026 00:04:00 GMT
• Title: Decomposing MXFP4 quantization error for LLM reinforcement learning: reducible bias, recoverable deadzone, and an irreducible floor
• Abstract summary: Existing work treats the quantization error as a monolithic noise term.<n>We prove an exact three-way decomposition of quantization error.<n>We show how each component dominates a distinct RL training pathway.
• Score: 2.5291809836356998
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: MXFP4 arithmetic can dramatically accelerate reinforcement learning (RL) post-training of large language models (LLMs), yet the quantization error introduces severe accuracy degradation. Existing work treats the quantization error as a monolithic noise term, missing the distinct mechanisms upon interpreting how quantization error damages training. We prove an exact three-way decomposition of quantization error and show how each component dominates a distinct RL training pathway. Our theoretical and empirical analysis decomposes the MXFP4 quantization error into three additive components: "scale bias" from power-of-two rounding, "deadzone truncation" from zeroing small values, and "grid noise" from rounding to the nearest 4-bit grid. Each component dominates a distinct RL failure mode: scale bias accumulates multiplicatively through the backward pass, affecting gradient accuracy; deadzone truncation degrades rollout quality; and grid noise raises the policy's entropy. We combine corrections that are RL failure mode-targeted but not component-exclusive: Macro-block scaling to reduce scale bias, Outlier Fallback recovers deadzone entries, but also partially reduces scale bias induced error, and Adaptive Quantization Noise (AQN) for controlling the policy entropy. On Qwen2.5-3B dense and Qwen3-30B-A3B-Base mixture-of-experts model, the targeted corrections recover BF16 accuracy to within 0.7% and exceed BF16 by +1.0% respectively.

Related papers

• OCGQuant: Outlier-Companion Grouping for NVFP4 Quantization [24.00034777837764]
  Outlier-Companion Grouping (OCG) adaptively pairs outlier channels with low-magnitude companion channels to improve NVFP4 activation block composition.<n>OCGQuant achieves the lowest WikiText-2 perplexity and highest average downstream accuracy among evaluated PTQ methods.
  arXiv  Detail & Related papers  (2026-08-30T16:00:11Z)
• Physics-Guided Linear Mapper for Quantum Error Mitigation [0.0]
  We introduce a novel physics-guided linear mapper (PGLM) for quantum error mitigation.<n>The goal is to provide a data-efficient, interpretable, and low-latency alternative to the black-box machine learning for quantum error mitigation.
  arXiv  Detail & Related papers  (2026-08-24T11:19:10Z)
• HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Models [9.149747042283197]
  We present the first end-to-end FP4 RL post-training, in which both the rollout and training policies operate at 4-bit precision.<n>A lightweight correction, Rollout-ResQ, recovers most of the precision lost to outlier-driven underflow without inflating the rollout's compute footprint.
  arXiv  Detail & Related papers  (2026-07-29T06:28:26Z)
• Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attention [34.07319488267708]
  Recent NVFP4 pretraining methods mainly target transformer linear layers, leaving computation states, arithmetic and attention underexplored in 4-bit pipelines.<n>This critical gap blocks stable full-stack 4-bit pretraining, as the three core modules exhibit unique numerical failure patterns.<n>We propose Full-Stack FP4, the first complete NVFP4 pretraining framework resolving all three stability bottlenecks.
  arXiv  Detail & Related papers  (2026-07-05T17:31:36Z)
• SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference [4.219937026460372]
  SharQ bridges activation sparsity and FP4 quantization through an online sparse--dense decomposition.<n>A sparse FP4 GEMM processes the backbone while a dense FP4 GEMM compensates for both mask-induced activation loss and sparse-path quantization error.
  arXiv  Detail & Related papers  (2026-06-25T04:19:04Z)
• TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization [13.984796236174331]
  We propose TORQ (Two-level Orthogonal Rotation for MXFP4 Quantization) as a training-free Post-Training Quantization framework.<n>We show that TORQ significantly improves the accuracy of MXFP4 activation quantization compared to existing methods.
  arXiv  Detail & Related papers  (2026-05-19T09:05:47Z)
• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution [3.6704226968275253]
  We show gradient-based methods that achieve meaningful forgetting lose it under compression, while methods that survive quantization barely change the model.<n>We present MANSU, which resolves both modes by combining causal circuit attribution to isolate the minimal forget-set subgraph.<n>We additionally introduce Circuit Attribution Divergence (CAD), a mechanistic verification metric distinguishing structural erasure from behavioral suppression.
  arXiv  Detail & Related papers  (2026-05-14T17:44:10Z)
• Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling [45.88028371034407]
  Post-training quantization (PTQ) is a practical path to deploy large diffusion models.<n>We propose Q-Drift, a principled sampler-side correction that treats quantization error as an implicit perturbation.<n>Q-Drift estimates a time-wise variance statistic from calibration, in practice requiring as few as 5 full-precision/quantized calibration runs.
  arXiv  Detail & Related papers  (2026-03-18T10:19:36Z)
• SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization [7.372706701787234]
  Post-training quantization (PTQ) has emerged as a prevailing technique for deploying large language models efficiently.<n>We propose SERQ, a saliency-aware error reconstruction method for low-bit LLM inference that employs a single low-rank compensation matrix.
  arXiv  Detail & Related papers  (2026-03-09T10:04:12Z)
• Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning [17.384089089363382]
  We identify a root cause that existing methods overlook: the uniform penalization of errors.<n>Current approaches treat all incorrect rollouts within a group identically.<n>We propose the Asymmetric Confidence-aware Error Penalty (ACE)
  arXiv  Detail & Related papers  (2026-02-24T22:46:43Z)
• CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal [84.71254539482369]
  Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures.<n>We present CARE, a failure-centric post-training framework for multimodal reasoning that turns errors into supervision.<n> CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures.
  arXiv  Detail & Related papers  (2025-12-22T16:34:21Z)
• CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training [73.46600457802693]
  We introduce a new method that counteracts the loss induced by quantization.<n>CAGE significantly improves upon the state-of-theart methods in terms of accuracy, for similar computational cost.<n>For QAT pre-training of Llama models, CAGE matches the accuracy achieved at 4-bits (W4A4) with the prior best method.
  arXiv  Detail & Related papers  (2025-10-21T16:33:57Z)
• LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning [50.89500210372827]
  Quantization and fine-tuning are crucial for deploying large language models (LLMs) on resource-constrained edge devices.<n>LoTA-QAF is a novel fine-tuning method specifically designed for quantized LLMs.<n>On the MMLU benchmark, our method effectively recovers performance for quantized models, surpassing 16-bit LoRA by up to 5.14%.
  arXiv  Detail & Related papers  (2025-05-24T14:47:28Z)
• Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners [51.32182730502002]
  We introduce Singular-value Diagonal Expansion to refine weight distributions to achieve better quantization alignment.<n>Our plug-and-play weight-quantization methods demonstrate substantial performance improvements over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-07-22T09:45:16Z)
• Towards Accurate Post-Training Quantization of Vision Transformers via Error Reduction [48.740630807085566]
  Post-training quantization (PTQ) for vision transformers (ViTs) has received increasing attention from both academic and industrial communities.<n>Current methods fail to account for the complex interactions between quantized weights and activations, resulting in significant quantization errors and suboptimal performance.<n>This paper presents ERQ, an innovative two-step PTQ method specifically crafted to reduce quantization errors arising from activation and weight quantization sequentially.
  arXiv  Detail & Related papers  (2024-07-09T12:06:03Z)
• The Accuracy vs. Sampling Overhead Trade-off in Quantum Error Mitigation Using Monte Carlo-Based Channel Inversion [84.66087478797475]
  Quantum error mitigation (QEM) is a class of promising techniques for reducing the computational error of variational quantum algorithms. We consider a practical channel inversion strategy based on Monte Carlo sampling, which introduces additional computational error. We show that when the computational error is small compared to the dynamic range of the error-free results, it scales with the square root of the number of gates.
  arXiv  Detail & Related papers  (2022-01-20T00:05:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.