Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate

• URL: http://arxiv.org/abs/2604.13806v1
• Date: Wed, 15 Apr 2026 12:40:30 GMT
• Title: Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate
• Abstract summary: Post-Training Quantization (PTQ) reduces memory footprint without retraining by leveraging a small calibration set.<n>Recent Hessian-based PTQ methods compensate quantization error via cross-channel dependencies, but such approaches degrade at low bit-widths.<n>We propose DASH-Q, a robust PTQ framework using diagonal Hessian approximation and iterative weighted least squares.
• Score: 9.512361190282261
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) are widely used across many domains, but their scale makes deployment challenging. Post-Training Quantization (PTQ) reduces memory footprint without retraining by leveraging a small calibration set. Recent Hessian-based PTQ methods compensate quantization error via cross-channel dependencies, but such approaches degrade at low bit-widths due to noisy curvature estimates from limited calibration data. We propose DASH-Q, a robust PTQ framework using diagonal Hessian approximation and iterative weighted least squares. By discarding noise-prone dependencies, DASH-Q filters sampling noise while prioritizing the preservation of salient feature power. We outperform other PTQ baselines in ultra low-bit regime, improving zero-shot accuracy by 7.01% on average and up to 14.01% over the strongest baselines across five baseline LLM models, while showing robust and stable performance with very small calibration data.

Related papers

• Recti-Q: Feature-Space Rectification for Out-of-Distribution-Robust Quantized Perception in Edge Robotics [18.66253365765404]
  We show that post-training quantization (PTQ) can substantially degrade reliability under deployment-relevant distribution shifts.<n>We propose Recti-Q, a lightweight feature-space rectification framework that freezes the quantized backbone and trains a small classifier-head LoRA adapter.<n>At less than 1% parameter overhead, Recti-Q preserves over 99% of PTQ memory savings, adds negligible compute, and enables low-bandwidth Over-The-Air resilience patching.
  arXiv  Detail & Related papers  (2026-07-20T22:14:04Z)
• Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models [8.779009700879982]
  Post-training quantization (PTQ) compresses deep neural networks for deployment under limited memory and computational budgets.<n>Most existing PTQ methods operate on an unconstrained full-precision (FP) model and primarily address quantization errors through post-hoc reconstruction.<n>We argue that low-bit PTQ accuracy is limited not only by post-quantization error minimization, but also by the quantization-error tolerance of FP model itself.
  arXiv  Detail & Related papers  (2026-07-13T10:24:36Z)
• Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models [50.16014098038291]
  Post-training quantization (PTQ) is critical for the efficient deployment of large language models (LLMs)<n>We propose SAGE-PTQ, a novel ultra-low-bit quantization framework for LLMs that minimizes hidden scaling cost.<n>On LLaMA-3B, SAGE-PTQ achieves 6.74 WikiText2 perplexity, compared to 55.8 for BiLLM, while using less than 50% of BiLLM's GPU memory.
  arXiv  Detail & Related papers  (2026-06-03T20:51:52Z)
• Saliency-Aware Regularized Quantization Calibration for Large Language Models [12.382772717031772]
  Post-training quantization (PTQ) is an effective approach for deploying large language models (LLMs) under memory and latency constraints.<n>Most existing PTQ methods determine quantization parameters by minimizing a layer-wise reconstruction error on a predetermined calibration dataset.<n>The proposed regularization encourages quantized weights to remain close to the original weights during calibration, leading to improved generalization at inference time.
  arXiv  Detail & Related papers  (2026-05-07T05:33:26Z)
• Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Sampling [45.88028371034407]
  Post-training quantization (PTQ) is a practical path to deploy large diffusion models.<n>We propose Q-Drift, a principled sampler-side correction that treats quantization error as an implicit perturbation.<n>Q-Drift estimates a time-wise variance statistic from calibration, in practice requiring as few as 5 full-precision/quantized calibration runs.
  arXiv  Detail & Related papers  (2026-03-18T10:19:36Z)
• Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models [41.677469535447024]
  Large Language Models (LLMs) deliver strong performance across a wide range of NLP tasks, but their massive sizes hinder deployment on resource-constrained devices.<n>Post-training quantization (PTQ) is widely adopted for its efficiency, as it requires no retraining and only a small dataset for calibration.<n>Recent advances for post-training quantization have demonstrated that even sub-4-bit methods can maintain most of the original model performance.
  arXiv  Detail & Related papers  (2025-12-25T12:39:36Z)
• Beyond Outliers: A Study of Optimizers Under Quantization [82.75879062804955]
  We study impact of choice on model robustness under quantization.<n>We evaluate how model performance degrades when trained with different baselines.<n>We derive scaling laws for quantization-aware training under different parameters.
  arXiv  Detail & Related papers  (2025-09-27T21:15:22Z)
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost [53.25965863436039]
  Quantization-aware training (QAT) provides a more principled solution, but its reliance on backpropagation incurs prohibitive memory costs.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework that supports both weight and activation quantization.<n>Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models [25.783531928577233]
  BASE-Q is a simple yet powerful approach that combines bias correction and asymmetric scaling to reduce rounding and clipping errors.<n>Experiments demonstrate the effectiveness of BASE-Q, narrowing the accuracy gap to full-precision models by 50.5%, 42.9%, and 29.2% compared to QuaRot, SpinQuant, and OSTQuant, respectively.
  arXiv  Detail & Related papers  (2025-05-26T14:22:21Z)
• PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models [64.84734437930362]
  Large Language Models (LLMs) suffer severe performance degradation when facing extremely low-bit (sub 2-bit) quantization.<n>We propose an extremely low-bit PTQ method called PTQ1.61, which enables weight quantization to 1.61-bit for the first time.<n>Experiments indicate our PTQ1.61 achieves state-of-the-art performance in extremely low-bit quantization.
  arXiv  Detail & Related papers  (2025-02-18T08:04:58Z)
• Rethinking Post-Training Quantization: Introducing a Statistical Pre-Calibration Approach [22.25748046511075]
  Post-training Quantization (PTQ) techniques rely on calibration processes to maintain their accuracy.<n>We propose a weight-adaptive PTQ method that can be considered a precursor to calibration-based PTQ methods.<n>We show that our proposed approach can perform on par with most common calibration-based PTQ methods.
  arXiv  Detail & Related papers  (2025-01-15T19:44:15Z)
• CBQ: Cross-Block Quantization for Large Language Models [66.82132832702895]
  Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.<n>We propose CBQ, a cross-block reconstruction-based PTQ method for LLMs.<n> CBQ employs a cross-block dependency using a reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation.
  arXiv  Detail & Related papers  (2023-12-13T07:56:27Z)
• Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance [53.45700148820669]
  Post-training quantization (PTQ) is a popular method for compressing deep neural networks (DNNs) without modifying their original architecture or training procedures. Despite its effectiveness and convenience, the reliability of PTQ methods in the presence of some extrem cases such as distribution shift and data noise remains largely unexplored. This paper first investigates this problem on various commonly-used PTQ methods.
  arXiv  Detail & Related papers  (2023-03-23T02:55:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.