BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization

• URL: http://arxiv.org/abs/2603.16590v1
• Date: Tue, 17 Mar 2026 14:37:08 GMT
• Title: BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization
• Abstract summary: Post-Training Quantization methods suffer from severe performance collapse when applied to MXFP4.<n>We propose BATQuant, which restricts transformations to align with MXFP to prevent cross-block outlier propagation.<n>We show that BATQuant establishes new state-of-the-art results under aggressive W4A4KV16 configurations.
• Score: 34.40883652676667
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Microscaling floating-point (MXFP) formats have emerged as a promising standard for deploying Multi-modal Large Language Models (MLLMs) and Large Language Models (LLMs) on modern accelerator architectures. However, existing Post-Training Quantization (PTQ) methods, particularly rotation-based techniques designed for integer formats, suffer from severe performance collapse when applied to MXFP4. Recent studies attribute this failure to a fundamental format mismatch: global orthogonal rotations inadvertently transfer outlier energy across quantization blocks, inducing new outliers that disrupt local block-wise scaling, while often creating bimodal activation distributions that underutilize the limited quantization range. To address these issues, we propose BATQuant (Block-wise Affine Transformation), which restricts transformations to align with MXFP granularity to prevent cross-block outlier propagation, while relaxing orthogonality constraints to optimize distribution shaping. To ensure parameter efficiency, we introduce Global and Private Kronecker (GPK) decomposition to effectively reduces storage and runtime overhead and incorporate Block-wise Learnable Clipping to suppress residual outliers. Extensive experiments on both MLLMs and LLMs demonstrate that BATQuant establishes new state-of-the-art results under aggressive W4A4KV16 configurations, recovering up to 96.43% of full-precision performance on multimodal benchmarks and clearly outperforming existing methods across diverse tasks.

Related papers

• Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMs [14.385983354036142]
  Low-bit quantization offers a promising avenue for reducing the computational and memory demands of Multimodal Large Language Models.<n>Recent hardware support for low-precision formats, such as MXFP8 and ultra-low-bit formats such as MXFP4 and HiF4, has accelerated research into efficient MLLM training and deployment.<n>We present a systematic study of these quantization schemes in representative MLLMs that span both video generation and reasoning tasks.
  arXiv  Detail & Related papers  (2026-08-27T03:47:58Z)
• MorphoQuant: Modality-Aware Quantization for Omni-modal Large Language Models [57.87791117741788]
  We propose a modality-aware PTQ framework to preserve cross-modal morphology and mitigate outlier loss.<n>Distribution-Aware Bias Compensation (DABC) selectively absorbs long-tailed outliers into channel-wise biases.<n>We also propose Morphology-Directed Quantization Function Optimization (MDQFO) to co-optimize the quantization grid with the bias mask.
  arXiv  Detail & Related papers  (2026-06-03T02:05:10Z)
• LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs [52.1276403258812]
  We introduce Logit-aware Final-block Quantization (LFQ), a simple yet effective enhancement to block-wise PTQ.<n>LFQ consistently improves the accuracy of complex generation tasks over state-of-the-art block-wise PTQ.
  arXiv  Detail & Related papers  (2026-05-28T11:02:23Z)
• Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention [58.484607892210015]
  Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation.<n>Many existing block-sparse attention methods select blocks by fixed sampling patterns over the high-resolution attention space.<n>We propose the Block Approximate Sparse Attention framework (BA-Att) with block-wise pre-downsampled operation.
  arXiv  Detail & Related papers  (2026-05-19T12:01:13Z)
• MAny: Merge Anything for Multimodal Continual Instruction Tuning [52.50936513604062]
  textbfMAny (textbfMAny) is a framework that merges task-specific knowledge through textbfCross-modal textbfProjection textbfMerging.<n>textbfLow-rank textbfParameter textbfMerging (textbfLPM)
  arXiv  Detail & Related papers  (2026-04-15T15:57:23Z)
• Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction [1.8599887993390827]
  Open Compute Project (OCP) Microscaling standard MXFP4 lags behind NVIDIA's NVFP4 in accuracy.<n>We introduce two software-only techniques, Overflow-Aware Scaling (OAS) and Macro Block Scaling (MBS)<n>OAS and MBS reduce the end-to-end accuracy gap between MXFP4 and NVFP4 from about 10% to below 1% on average.
  arXiv  Detail & Related papers  (2026-01-30T23:24:17Z)
• ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs [4.431548809730958]
  ARCQuant is a framework that boosts NVFP4 performance via Augmented Residual Channels.<n>We show that ARCQuant achieves state-of-the-art accuracy, comparable to full-precision baselines in perplexity and downstream tasks.
  arXiv  Detail & Related papers  (2026-01-12T12:27:22Z)
• MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving [4.176741972965246]
  Reduced-precision data formats are crucial for cost-effective serving of large language models (LLMs)<n>In this paper, we focus on recent industry-driven variants of block floating-point (BFP) formats.<n>We propose MX+, a cost-effective and non-intrusive extension designed for seamless integration into the microscaling (MX) formats.
  arXiv  Detail & Related papers  (2025-10-16T11:05:54Z)
• I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models [20.070306492164427]
  Post-training quantization serves as a potent technique to accelerate the inference of large language models. Existing works still necessitate a considerable number of floating-point (FP) operations during inference. This limitation hinders the deployment of large language models on the edge and cloud devices. We propose I-LLM, a novel integer-only fully-quantized PTQ framework tailored for large language models.
  arXiv  Detail & Related papers  (2024-05-28T05:56:11Z)
• DB-LLM: Accurate Dual-Binarization for Efficient LLMs [83.70686728471547]
  Large language models (LLMs) have significantly advanced the field of natural language processing. Existing ultra-low-bit quantization always causes severe accuracy drops. We propose a novel Dual-Binarization method for LLMs, namely DB-LLM.
  arXiv  Detail & Related papers  (2024-02-19T09:04:30Z)
• CBQ: Cross-Block Quantization for Large Language Models [66.82132832702895]
  Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.<n>We propose CBQ, a cross-block reconstruction-based PTQ method for LLMs.<n> CBQ employs a cross-block dependency using a reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation.
  arXiv  Detail & Related papers  (2023-12-13T07:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.