The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs

• URL: http://arxiv.org/abs/2607.08734v1
• Date: Thu, 09 Jul 2026 17:35:02 GMT
• Title: The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
• Abstract summary: Post-training quantization is widely used to deploy large language models in resource-constrained settings.<n>We show that these metrics fail to capture behavioral changes induced by quantization.<n>We introduce correctness agreement, a decision-level metric that measures overlap in correct predictions between a base model and its quantized variants.
• Score: 5.849736173068869
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-training quantization is widely used to deploy large language models in resource-constrained settings, yet its evaluation relies almost exclusively on accuracy and perplexity. We show that these metrics fail to capture behavioral changes induced by quantization. We introduce correctness agreement, a decision-level metric that measures overlap in correct predictions between a base model and its quantized variants, independent of absolute accuracy. Across multiple models and quantization schemes from 8-bit to 2-bit, we find that behavioral divergence emerges under moderate quantization even when task performance appears preserved. To explain this effect, we analyze quantization as a structural operator on attention weights and quantify layer-wise distortions using statistical and distributional measures. Our results reveal non-linear breakpoints at low bit-widths and show that query and key projections are consistently more sensitive than value and output projections. These findings expose an illusion of equivalence between base and quantized models and motivate behavioral evaluation beyond conventional performance metrics.

Related papers

• Accuracy is Not Enough: A Divergence-Based Approach to Evaluate Fidelity Loss in Quantized LLMs [13.897073114465094]
  Deployment of Large Language Models (LLMs) on memory-constrained edge devices relies heavily on aggressive post-training quantization.<n>We introduce a distribution-sensitive evaluation framework quantifying information loss in quantized LLMs.<n>We quantify probability mass displacement and distributional drift relative to the BF16 reference, capturing predictive distribution changes not reflected in top-1 accuracy.
  arXiv  Detail & Related papers  (2026-09-07T15:49:52Z)
• Quantization Degradation in Large Language Models: A Signal-Noise Perspective [31.171218460791483]
  We systematically study weight-only post-training quantization across bit-widths, quantization methods, model scales and downstream tasks.<n>We observe that such degradation varies substantially across these factors.<n>We use the signal-to-noise ratio (SNR) to measure how strongly quantization perturbs full-precision representations.
  arXiv  Detail & Related papers  (2026-08-08T15:28:45Z)
• Reliability Scaling Laws for Quantized Large Language Models [68.39028482878265]
  Quantization is a strategy to build capable and resource-efficient large language models (LLMs) by reducing the bitwidth of the parameters.<n>We first conduct a comprehensive reliability evaluation of quantized LLMs consisting of three key components.<n>A reliability peak occurs for 4-bit quantized models, indicating that quantizing moderately sized models offers the best reliability-efficiency trade-off.
  arXiv  Detail & Related papers  (2026-07-12T17:39:09Z)
• Scaling Laws for Precision in High-Dimensional Linear Regression [38.87908801454087]
  We study scaling laws for low-precision training within a high-dimensional sketched linear regression framework.<n>By analyzing multiplicative and additive quantization, we identify a critical dichotomy in their scaling behaviors.<n>Our work provides a theoretical basis for optimizing training protocols under practical hardware constraints.
  arXiv  Detail & Related papers  (2026-02-22T15:51:29Z)
• Explaining How Quantization Disparately Skews a Model [8.210473195536077]
  Post Training Quantization (PTQ) is widely adopted due to its high compression capacity and speed with minimal impact on accuracy.<n>We observed that disparate impacts are exacerbated by quantization, especially for minority groups.<n>We explore how the changes in weights and activations induced by quantization cause cascaded impacts in the network, resulting in logits with lower variance, increased loss, and compromised group accuracies.
  arXiv  Detail & Related papers  (2025-09-08T21:04:16Z)
• How Quantization Shapes Bias in Large Language Models [61.40435736418359]
  We focus on weight and activation quantization strategies and examine their effects across a broad range of bias types.<n>We employ both probabilistic and generated text-based metrics across nine benchmarks and evaluate models varying in architecture family and reasoning ability.
  arXiv  Detail & Related papers  (2025-08-25T14:48:26Z)
• Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners [51.32182730502002]
  We introduce Singular-value Diagonal Expansion to refine weight distributions to achieve better quantization alignment.<n>Our plug-and-play weight-quantization methods demonstrate substantial performance improvements over state-of-the-art approaches.
  arXiv  Detail & Related papers  (2024-07-22T09:45:16Z)
• When Quantization Affects Confidence of Large Language Models? [4.338589334157708]
  We show that GPTQ to 4-bit results in a decrease in confidence regarding true labels, with varying impacts observed among different language models. We propose an explanation for quantization loss based on confidence levels, indicating that quantization disproportionately affects samples where the full model exhibited low confidence levels in the first place.
  arXiv  Detail & Related papers  (2024-05-01T16:58:28Z)
• Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study [90.34226812493083]
  This work aims to investigate the impact of quantization on emphemergent abilities, which are important characteristics that distinguish LLMs from small language models. Our empirical experiments show that these emergent abilities still exist in 4-bit quantization models, while 2-bit models encounter severe performance degradation. To improve the performance of low-bit models, we conduct two special experiments: (1) fine-gained impact analysis that studies which components (or substructures) are more sensitive to quantization, and (2) performance compensation through model fine-tuning.
  arXiv  Detail & Related papers  (2023-07-16T15:11:01Z)
• Regularized Vector Quantization for Tokenized Image Synthesis [126.96880843754066]
  Quantizing images into discrete representations has been a fundamental problem in unified generative modeling. deterministic quantization suffers from severe codebook collapse and misalignment with inference stage while quantization suffers from low codebook utilization and reconstruction objective. This paper presents a regularized vector quantization framework that allows to mitigate perturbed above issues effectively by applying regularization from two perspectives.
  arXiv  Detail & Related papers  (2023-03-11T15:20:54Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• An Investigation on Different Underlying Quantization Schemes for Pre-trained Language Models [33.49417100179159]
  We implement k-means quantization and compare its performance on the fix-precision quantization of BERT with linear quantization. We also compare the two quantization schemes on ALBERT models to explore the robustness differences between different pre-trained models.
  arXiv  Detail & Related papers  (2020-10-14T14:05:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.