ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization

• URL: http://arxiv.org/abs/2608.07019v1
• Date: Fri, 07 Aug 2026 09:27:14 GMT
• Title: ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
• Abstract summary: Post-training quantization (PTQ) is widely used to reduce the computational cost of large language models.<n>We introduce ReQuant, a backpropagation-free fixed-grid refinement procedure for this stage.<n>We show that ReQuant consistently improves quantized models from heterogeneous PTQ initializers.
• Score: 10.647335642551193
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Post-training quantization (PTQ) is widely used to reduce the memory and computational cost of large language models. Existing PTQ methods typically obtain an initial quantized model through heuristic rules or greedy optimization, and once quantization is completed the resulting integer assignments are usually treated as final. This observation motivates a complementary optimization stage within PTQ that keeps quantized weights improvable after an executable quantized model has been produced, while preserving the quantized format. We introduce ReQuant, a backpropagation-free fixed-grid refinement procedure for this stage. Agnostic to the PTQ initializer, ReQuant takes an existing quantized model as a feasible starting point and iteratively revisits its discrete weight assignments on the fixed quantization grid. Accepted updates strictly reduce the mean squared reconstruction error and remain on the original grid. In this way, ReQuant turns the initially fixed PTQ output into an iteratively optimizable discrete solution and serves as a plug-and-play post-processing stage for existing PTQ pipelines. Experiments across diverse model families, bit-widths, and downstream tasks show that ReQuant consistently improves quantized models from heterogeneous PTQ initializers, with especially large gains on simple initializers and lower bit-widths. Notably, ReQuant can refine a simple round-to-nearest initialization across multiple sweeps until it approaches or surpasses GPTAQ under the same quantization format. These results establish ReQuant as a practical complementary stage for further improving existing PTQ pipelines.

Related papers

• ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers [60.09590321091875]
  Post-Training Quantization (PTQ) offers an attractive solution by compressing models using a small calibration set with minimal training overhead.<n>Most existing PTQ works adopt a static quantization paradigm that is uniformly applied to all instances.<n>We propose ScalePredictor, a dynamic quantization framework for accurate and efficient quantization scale learning of ViTs.
  arXiv  Detail & Related papers  (2026-06-20T08:33:55Z)
• Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin [13.185423069216808]
  Post-training quantization converts a trained full-precision model into low-bit weights without task-level retraining.<n>Although PTQ is efficient and often accurate at moderate bitwidths, it can fail sharply at aggressive bitwidths.<n>We propose a unified geometric framework that explains both PTQ failure and QAT recovery.
  arXiv  Detail & Related papers  (2026-06-08T04:21:39Z)
• Saliency-Aware Regularized Quantization Calibration for Large Language Models [12.382772717031772]
  Post-training quantization (PTQ) is an effective approach for deploying large language models (LLMs) under memory and latency constraints.<n>Most existing PTQ methods determine quantization parameters by minimizing a layer-wise reconstruction error on a predetermined calibration dataset.<n>The proposed regularization encourages quantized weights to remain close to the original weights during calibration, leading to improved generalization at inference time.
  arXiv  Detail & Related papers  (2026-05-07T05:33:26Z)
• SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Models [35.243028473008685]
  We propose a simple and effective post-training quantization paradigm for large language models (LLMs)<n> SEPTQ first calculates the importance score for each element in the weight matrix and determines the quantization locations in a static global manner.<n>Compared with previous methods, SEPTQ simplifies the post-training quantization procedure into only two steps, and considers the effectiveness and efficiency simultaneously.
  arXiv  Detail & Related papers  (2026-04-11T08:23:50Z)
• LoaQ: Layer-wise Output Approximation Quantization [40.425536766574844]
  A natural idea in model quantization is to approximate each component's quantized output to match its original.<n>Layer-wise post-training quantization (PTQ) can achieve, at best, only activation-aware approximations of weights.<n>We propose $LoaQ$, an output-approximation method for layer-wise PTQ that explicitly targets output-level consistency.
  arXiv  Detail & Related papers  (2025-09-08T02:50:11Z)
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost [53.25965863436039]
  Quantization-aware training (QAT) provides a more principled solution, but its reliance on backpropagation incurs prohibitive memory costs.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework that supports both weight and activation quantization.<n>Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation [74.34220141721231]
  We present MPQ-DMv2, an improved textbfMixed textbfPrecision textbfQuantization framework for extremely low-bit textbfDiffusion textbfModels.
  arXiv  Detail & Related papers  (2025-07-06T08:16:50Z)
• FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation [55.12070409045766]
  Post-training quantization (PTQ) has stood out as a cost-effective and promising model compression paradigm in recent years.<n>Current PTQ methods for Vision Transformers (ViTs) still suffer from significant accuracy degradation, especially under low-bit quantization.
  arXiv  Detail & Related papers  (2025-06-13T07:57:38Z)
• APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers [71.2294205496784]
  We propose textbfAPHQ-ViT, a novel PTQ approach based on importance estimation with Average Perturbation Hessian (APH)<n>We show that APHQ-ViT using linear quantizers outperforms existing PTQ methods by substantial margins in 3-bit and 4-bit across different vision tasks.
  arXiv  Detail & Related papers  (2025-04-03T11:48:56Z)
• PreQuant: A Task-agnostic Quantization Approach for Pre-trained Language Models [52.09865918265002]
  We propose a novel quantize before fine-tuning'' framework, PreQuant. PreQuant is compatible with various quantization strategies, with outlier-aware fine-tuning incorporated to correct the induced quantization error. We demonstrate the effectiveness of PreQuant on the GLUE benchmark using BERT, RoBERTa, and T5.
  arXiv  Detail & Related papers  (2023-05-30T08:41:33Z)
• Distribution-Flexible Subset Quantization for Post-Quantizing Super-Resolution Networks [68.83451203841624]
  This paper introduces Distribution-Flexible Subset Quantization (DFSQ), a post-training quantization method for super-resolution networks. DFSQ conducts channel-wise normalization of the activations and applies distribution-flexible subset quantization (SQ) It achieves comparable performance to full-precision counterparts on 6- and 8-bit quantization, and incurs only a 0.1 dB PSNR drop on 4-bit quantization.
  arXiv  Detail & Related papers  (2023-05-10T04:19:11Z)
• QFT: Post-training quantization via fast joint finetuning of all degrees of freedom [1.1744028458220428]
  We rethink quantized network parameterization in HW-aware fashion, towards a unified analysis of all quantization DoF. Our single-step simple and extendable method, dubbed quantization-aware finetuning (QFT), achieves 4-bit weight quantization results on-par with SoTA.
  arXiv  Detail & Related papers  (2022-12-05T22:38:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.