SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization

• URL: http://arxiv.org/abs/2608.15567v1
• Date: Sun, 16 Aug 2026 06:29:10 GMT
• Title: SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
• Abstract summary: We introduce SCHUROPT, which analytically eliminates the suffix's optimal continuous response.<n>It alternates closed-form row-wise scale/zero-point refitting with coordinate descent over integer codes.<n>Across eight Llama and Qwen models, SCHURQUANT achieves the highest mean zero-shot accuracy.
• Score: 9.043029012704817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Weight-only post-training quantization (PTQ) enables the deployment of large language models under tight memory budgets, but accuracy often collapses at 2-3 bits. Existing backpropagation-free PTQ optimizers have two limitations: group decisions ignore the correction that the remaining continuous suffix can absorb, and discrete refinements typically keep the affine quantization grid fixed. We introduce SCHUROPT, which analytically eliminates the suffix's optimal continuous response, yielding an exact groupwise quadratic with Schur-complement curvature. It then alternates closed-form row-wise scale/zero-point refitting with coordinate descent over integer codes. With the GPTQ objective fixed, SCHUROPT improves mean zero-shot accuracy on 2-bit Qwen3-4B by 11.88 percentage points (pp). At higher precision, however, tighter reconstruction does not consistently improve end-model metrics. SCHURQUANT therefore combines SCHUROPT with quantized-prefix teacher reconstruction, reference-weight regularization, residual-add targets, and teacher-decision token weighting. Across eight Llama and Qwen models, SCHURQUANT achieves the highest mean zero-shot accuracy among the evaluated backpropagation free PTQ baselines, outperforming the strongest baseline by 9.65 pp at 2 bits.

Related papers

• Recurrent Residual Quantization: A Progressive Multi-Precision Representation for LLMs [9.68624792431081]
  We introduce Recurrent Residual Quantization (RRQ), a post-training quantization framework.<n>RRQ represents weights as a low-bit quantized base together with a sequence of quantized residual corrections.<n>In our Qwen3-8B setup, the full all-RTN 2-/4-/6-/8-bit package is constructed in 1,293 seconds, 3.3 times faster than the measured MatGPTQ construction.
  arXiv  Detail & Related papers  (2026-08-04T08:32:05Z)
• LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMs [52.1276403258812]
  We introduce Logit-aware Final-block Quantization (LFQ), a simple yet effective enhancement to block-wise PTQ.<n>LFQ consistently improves the accuracy of complex generation tasks over state-of-the-art block-wise PTQ.
  arXiv  Detail & Related papers  (2026-05-28T11:02:23Z)
• Saliency-Aware Regularized Quantization Calibration for Large Language Models [12.382772717031772]
  Post-training quantization (PTQ) is an effective approach for deploying large language models (LLMs) under memory and latency constraints.<n>Most existing PTQ methods determine quantization parameters by minimizing a layer-wise reconstruction error on a predetermined calibration dataset.<n>The proposed regularization encourages quantized weights to remain close to the original weights during calibration, leading to improved generalization at inference time.
  arXiv  Detail & Related papers  (2026-05-07T05:33:26Z)
• BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models [56.504879072674015]
  We propose Bit-Plane Decomposition Quantization (BPDQ), which constructs a variable quantization grid via bit-planes and scalar coefficients.<n>BPDQ enables serving Qwen2.5-72B on a single GTX 3090 with 83.85% GSM8K accuracy (vs. 90.83% at 16-bit)
  arXiv  Detail & Related papers  (2026-02-04T02:54:37Z)
• ECO: Quantized Training without Full-Precision Master Weights [58.97082407934466]
  Error-Compensating (ECO) eliminates master weights by applying updates directly to quantized parameters.<n>We show that ECO converges to a constant-radius neighborhood of the optimum, while naive master-weight removal can incur an error that is inversely proportional to the learning rate.
  arXiv  Detail & Related papers  (2026-01-29T18:35:01Z)
• LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotation [6.797237769820339]
  Post-training quantization (PTQ) enables effective model compression while preserving relatively high accuracy.<n>We propose LoPRo, a novel fine-tuning-free PTQ algorithm that enhances residual matrix quantization.<n> Experiments demonstrate that LoPRo outperforms existing fine-tuning-free PTQ methods at both 2-bit and 3-bit quantization.
  arXiv  Detail & Related papers  (2026-01-27T14:56:04Z)
• BayesQ: Uncertainty-Guided Bayesian Quantization [1.565870461096057]
  BayesQ is an uncertainty-guided post-training quantization framework.<n>It is the first to optimize quantization under the posterior expected loss.<n>BayesQ reframes low-bit quantization as uncertainty-aware risk minimization.
  arXiv  Detail & Related papers  (2025-11-11T22:36:19Z)
• End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost [53.25965863436039]
  Quantization-aware training (QAT) provides a more principled solution, but its reliance on backpropagation incurs prohibitive memory costs.<n>We propose ZeroQAT, a zeroth-order optimization-based QAT framework that supports both weight and activation quantization.<n>Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory.
  arXiv  Detail & Related papers  (2025-08-21T01:18:27Z)
• LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning [50.89500210372827]
  Quantization and fine-tuning are crucial for deploying large language models (LLMs) on resource-constrained edge devices.<n>LoTA-QAF is a novel fine-tuning method specifically designed for quantized LLMs.<n>On the MMLU benchmark, our method effectively recovers performance for quantized models, surpassing 16-bit LoRA by up to 5.14%.
  arXiv  Detail & Related papers  (2025-05-24T14:47:28Z)
• QERA: an Analytical Framework for Quantization Error Reconstruction [12.110441045050223]
  An increasing interest in quantizing weights to extremely low precision while offsetting the resulting error with low-rank, high-precision error reconstruction terms.<n>The combination of quantization and low-rank approximation is now popular in both adapter-based, parameter-efficient fine-tuning methods.<n>We formulate an analytical framework, named Quantization Error Reconstruction Analysis (QERA), and offer a closed-form solution to the problem.
  arXiv  Detail & Related papers  (2024-10-08T13:37:34Z)
• SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models [63.118592279833656]
  Post-training quantization (PTQ) is an effective technique for compressing large language models (LLMs)<n>We propose SliM-LLM, a salience-driven mixed-precision quantization framework that allocates bit-widths at the group-wise.<n> Experiments show that SliM-LLM achieves superior performance across various LLMs at low bit-widths.
  arXiv  Detail & Related papers  (2024-05-23T16:21:48Z)
• CBQ: Cross-Block Quantization for Large Language Models [66.82132832702895]
  Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.<n>We propose CBQ, a cross-block reconstruction-based PTQ method for LLMs.<n> CBQ employs a cross-block dependency using a reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation.
  arXiv  Detail & Related papers  (2023-12-13T07:56:27Z)
• CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification [51.81850995661478]
  Mixed-precision quantization has been widely applied on deep neural networks (DNNs) Previous attempts on bit-level regularization and pruning-based dynamic precision adjustment during training suffer from noisy gradients and unstable convergence. We propose Continuous Sparsification Quantization (CSQ), a bit-level training method to search for mixed-precision quantization schemes with improved stability.
  arXiv  Detail & Related papers  (2022-12-06T05:44:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.