Dissecting Quantization Error: A Concentration-Alignment Perspective

• URL: http://arxiv.org/abs/2603.04359v1
• Date: Wed, 04 Mar 2026 18:26:24 GMT
• Title: Dissecting Quantization Error: A Concentration-Alignment Perspective
• Authors: Marco Federici, Boris van Breugel, Paul Whatmough, Markus Nagel,
• Abstract summary: We analyze linear-layer quantization via the signal-to-quantization-noise ratio (SQNR)<n>We show that for uniform integer quantization at a fixed bit width, SQNR decomposes into (i) the concentration of weights and activations, and (ii) the alignment of their dominant variation directions.<n>Motivated by this, we introduce block Concentration-Alignment Transforms (CAT), a lightweight linear transformation that uses a covariance estimate from a small calibration set to jointly improve concentration and alignment.
• Score: 23.355895097785083
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Quantization can drastically increase the efficiency of large language and vision models, but typically incurs an accuracy drop. Recently, function-preserving transforms (e.g. rotations, Hadamard transform, channel-wise scaling) have been successfully applied to reduce post-training quantization error, yet a principled explanation remains elusive. We analyze linear-layer quantization via the signal-to-quantization-noise ratio (SQNR), showing that for uniform integer quantization at a fixed bit width, SQNR decomposes into (i) the concentration of weights and activations (capturing spread and outliers), and (ii) the alignment of their dominant variation directions. This reveals an actionable insight: beyond concentration - the focus of most prior transforms (e.g. rotations or Hadamard) - improving alignment between weight and activation can further reduce quantization error. Motivated by this, we introduce block Concentration-Alignment Transforms (CAT), a lightweight linear transformation that uses a covariance estimate from a small calibration set to jointly improve concentration and alignment, approximately maximizing SQNR. Experiments across several LLMs show that CAT consistently matches or outperforms prior transform-based quantization methods at 4-bit precision, confirming the insights gained in our framework.

Related papers

• On the Importance of a Multi-Scale Calibration for Quantization [9.237936671153749]
  Post-training quantization (PTQ) is a cornerstone for efficiently deploying large language models (LLMs)<n>We propose MaCa (Matryoshka), a simple yet effective method for length-aware Hessian construction.<n> Experiments on state-of-the-art LLMs (e.g., Qwen3, Gemma3, LLaMA3) demonstrate that MaCa consistently improves accuracy under low bit quantization.
  arXiv  Detail & Related papers  (2026-02-07T09:42:53Z)
• WUSH: Near-Optimal Adaptive Transforms for LLM Quantization [52.77441224845925]
  Quantization to low bitwidth is a standard approach for deploying large language models.<n>A few extreme weights and activations stretch the dynamic range and reduce the effective resolution of the quantizer.<n>We derive, for the first time, closed-form optimal linear blockwise transforms for joint weight-activation quantization.
  arXiv  Detail & Related papers  (2025-11-30T16:17:34Z)
• STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization [21.93314755695813]
  Quantization is the key method for reducing inference latency, power and memory footprint of generative AI models.<n>We propose textitSequence Transformation and Mixed Precision (STaMP) quantization.
  arXiv  Detail & Related papers  (2025-10-30T17:53:42Z)
• Turning LLM Activations Quantization-Friendly [0.0]
  Quantization effectively reduces the serving costs of Large Language Models (LLMs) by speeding up data movement through compressed parameters and enabling faster operations via integer arithmetic.<n>However, activating integer arithmetic requires quantizing both weights and activations, which poses challenges due to the significant outliers in LLMs that increase quantization error.<n>In this work, we investigate these outliers with an emphasis on their effect on layer-wise quantization error, then examine how smoothing and rotation transform the observed values.
  arXiv  Detail & Related papers  (2025-05-11T17:13:55Z)
• RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models [53.571195477043496]
  We propose an algorithm named Rotated Straight-Through-Estimator (RoSTE)<n>RoSTE combines quantization-aware supervised fine-tuning (QA-SFT) with an adaptive rotation strategy to reduce activation outliers.<n>Our findings reveal that the prediction error is directly proportional to the quantization error of the converged weights, which can be effectively managed through an optimized rotation configuration.
  arXiv  Detail & Related papers  (2025-02-13T06:44:33Z)
• FlatQuant: Flatness Matters for LLM Quantization [58.28221892035609]
  We propose FlatQuant, a new post-training quantization approach that enhances the flatness of weights and activations.<n>Our approach identifies optimal affine transformations for each linear layer, calibrated in hours via a lightweight objective.<n>It achieves less than 1% accuracy drop for W4A4 quantization on the LLaMA-3-70B model, surpassing SpinQuant by 7.5%.
  arXiv  Detail & Related papers  (2024-10-12T08:10:28Z)
• QT-DoG: Quantization-aware Training for Domain Generalization [58.439816306817306]
  We propose Quantization-aware Training for Domain Generalization (QT-DoG)<n>We demonstrate that weight quantization effectively leads to flatter minima in the loss landscape.<n> QT-DoG exploits quantization as an implicit regularizer by inducing noise in model weights.
  arXiv  Detail & Related papers  (2024-10-08T13:21:48Z)
• Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners [51.32182730502002]
  We introduce Singular-value Diagonal Expansion to refine weight distributions to achieve better quantization alignment.<n>Our plug-and-play weight-quantization methods demonstrate substantial performance improvements over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-07-22T09:45:16Z)
• Towards Accurate Post-Training Quantization of Vision Transformers via Error Reduction [48.740630807085566]
  Post-training quantization (PTQ) for vision transformers (ViTs) has received increasing attention from both academic and industrial communities.<n>Current methods fail to account for the complex interactions between quantized weights and activations, resulting in significant quantization errors and suboptimal performance.<n>This paper presents ERQ, an innovative two-step PTQ method specifically crafted to reduce quantization errors arising from activation and weight quantization sequentially.
  arXiv  Detail & Related papers  (2024-07-09T12:06:03Z)
• Where Should We Begin? A Low-Level Exploration of Weight Initialization Impact on Quantized Behaviour of Deep Neural Networks [93.4221402881609]
  We present an in-depth, fine-grained ablation study of the effect of different weights initialization on the final distributions of weights and activations of different CNN architectures. To our best knowledge, we are the first to perform such a low-level, in-depth quantitative analysis of weights initialization and its effect on quantized behaviour.
  arXiv  Detail & Related papers  (2020-11-30T06:54:28Z)
• AUSN: Approximately Uniform Quantization by Adaptively Superimposing Non-uniform Distribution for Deep Neural Networks [0.7378164273177589]
  Existing uniform and non-uniform quantization methods exhibit an inherent conflict between the representing range and representing resolution. We propose a novel quantization method to quantize the weight and activation. The key idea is to Approximate the Uniform quantization by Adaptively Superposing multiple Non-uniform quantized values, namely AUSN.
  arXiv  Detail & Related papers  (2020-07-08T05:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.