DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

• URL: http://arxiv.org/abs/2606.12487v1
• Date: Wed, 10 Jun 2026 09:25:45 GMT
• Title: DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics
• Abstract summary: Post-training quantization (PTQ) is essential for efficient large language model inference.<n>A key difficulty lies in massive activations, whose extreme values dominate the activation range and amplify quantization errors.<n>We propose DynamicPTQ, a Dynamic Post-Training Quantization policy for phase-aware mixed-precision activation quantization.
• Score: 21.115748726323606
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Post-training quantization (PTQ) is essential for efficient large language model inference, but reliably quantizing activations remains challenging when weights, activations, and KV caches are all quantized to 4-bit precision. A key difficulty lies in massive activations, whose extreme values dominate the activation range and amplify quantization errors. State-of-the-art methods mainly mitigate massive activations through transformation-based smoothing, such as orthogonal rotations and affine scaling, but overlook the cross-layer dynamics of the residual stream. In this paper, we show that massive activations emerge and disappear in a phase-wise pattern across network depth, triggering large residual changes. These changes cause newly injected layer-wise updates to dominate the 4-bit quantization scale and weaken historical residual information. To characterize this behavior, we introduce Jump Ratio and Historical Feature SNR. This suggests that static transformation-based smoothing cannot fully resolve dynamic quantization instability caused by cross-layer residual changes. Based on this analysis, we propose DynamicPTQ, a Dynamic Post-Training Quantization policy for phase-aware mixed-precision activation quantization. DynamicPTQ identifies quantization-sensitive layers from residual-stream dynamics and assigns 8-bit activation precision only to these layers, while keeping weights, KV caches, and other activations in 4-bit precision. It can be directly integrated with strong PTQ baselines such as QuaRot, SpinQuant, and FlatQuant. Experiments on LLaMA-2 and LLaMA-3 show that DynamicPTQ consistently improves perplexity and zero-shot QA performance under W4A4KV4 quantization, while achieving 1.05 to 1.07 times throughput improvement with modest memory overhead. These results demonstrate a practical path toward robust low-bit LLM inference.

Related papers

• Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMs [14.385983354036142]
  Low-bit quantization offers a promising avenue for reducing the computational and memory demands of Multimodal Large Language Models.<n>Recent hardware support for low-precision formats, such as MXFP8 and ultra-low-bit formats such as MXFP4 and HiF4, has accelerated research into efficient MLLM training and deployment.<n>We present a systematic study of these quantization schemes in representative MLLMs that span both video generation and reasoning tasks.
  arXiv  Detail & Related papers  (2026-08-27T03:47:58Z)
• PTQ4SNN: Membrane-Aware Post-Training Quantization for Spiking Neural Networks [51.21235018272886]
  Spiking neural networks (SNNs) enable sparse and event-driven computation, but their low-bit deployment remains incomplete.<n>We propose PTQ4SNN, a membrane-aware post-training quantization framework that jointly quantizes weights and recurrent membrane states using only a small calibration set.<n>Experiments on static and event-based classification and semantic segmentation show that PTQ4SNN effectively preserves model accuracy under W4 quantization and approximately 4-bit membrane precision.
  arXiv  Detail & Related papers  (2026-08-07T10:16:25Z)
• ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers [60.09590321091875]
  Post-Training Quantization (PTQ) offers an attractive solution by compressing models using a small calibration set with minimal training overhead.<n>Most existing PTQ works adopt a static quantization paradigm that is uniformly applied to all instances.<n>We propose ScalePredictor, a dynamic quantization framework for accurate and efficient quantization scale learning of ViTs.
  arXiv  Detail & Related papers  (2026-06-20T08:33:55Z)
• LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotation [6.797237769820339]
  Post-training quantization (PTQ) enables effective model compression while preserving relatively high accuracy.<n>We propose LoPRo, a novel fine-tuning-free PTQ algorithm that enhances residual matrix quantization.<n> Experiments demonstrate that LoPRo outperforms existing fine-tuning-free PTQ methods at both 2-bit and 3-bit quantization.
  arXiv  Detail & Related papers  (2026-01-27T14:56:04Z)
• STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization [21.93314755695813]
  Quantization is the key method for reducing inference latency, power and memory footprint of generative AI models.<n>We propose textitSequence Transformation and Mixed Precision (STaMP) quantization.
  arXiv  Detail & Related papers  (2025-10-30T17:53:42Z)
• Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigation [18.963480523099694]
  Spiking Neural Networks (SNNs) support mixed-precision storage and energy-efficient computation by replacing complex MACs with temporal Accumulate (ACCs)<n>We propose SpikeQuant, which selectively applies mixed-precision quantization to activations with salient values and re-encodes them into binary spike counts.<n> Experimental results demonstrate that SpikeQuant consistently achieves near-FP16 perplexity under W4A4 quantization while reducing energy cost by up to 4.6 times compared to existing methods.
  arXiv  Detail & Related papers  (2025-10-22T11:50:00Z)
• FlatQuant: Flatness Matters for LLM Quantization [58.28221892035609]
  We propose FlatQuant, a new post-training quantization approach that enhances the flatness of weights and activations.<n>Our approach identifies optimal affine transformations for each linear layer, calibrated in hours via a lightweight objective.<n>It achieves less than 1% accuracy drop for W4A4 quantization on the LLaMA-3-70B model, surpassing SpinQuant by 7.5%.
  arXiv  Detail & Related papers  (2024-10-12T08:10:28Z)
• PTQ4DiT: Post-training Quantization for Diffusion Transformers [52.902071948957186]
  Post-training Quantization (PTQ) has emerged as a fast and data-efficient solution that can significantly reduce computation and memory footprint. We propose PTQ4DiT, a specifically designed PTQ method for DiTs. We demonstrate that our PTQ4DiT successfully quantizes DiTs to 8-bit precision while preserving comparable generation ability.
  arXiv  Detail & Related papers  (2024-05-25T02:02:08Z)
• Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization [62.15918574997175]
  It is known that language models contain outlier channels whose values on average are orders of magnitude higher than other channels. We propose a strategy which regularizes a layer's inputs via quantization-aware training (QAT) and its outputs via activation kurtosis regularization. We show that regularizing both the inputs and outputs is crucial for preventing a model's "migrating" the difficulty in input quantization to the weights.
  arXiv  Detail & Related papers  (2024-04-04T17:25:30Z)
• BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation [116.26521375592759]
  Quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation. Extreme quantization (1-bit weight/1-bit activations) of compactly-designed backbone architectures results in severe performance degeneration. This paper proposes a novel Quantization-Aware Training (QAT) method that can effectively alleviate performance degeneration.
  arXiv  Detail & Related papers  (2022-07-04T13:25:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.