ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs

• URL: http://arxiv.org/abs/2605.10793v1
• Date: Mon, 11 May 2026 16:23:10 GMT
• Title: ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs
• Abstract summary: Large language models (LLMs) are costly to deploy due to their large memory footprint and high inference cost.<n>Weight-activation quantization can reduce these costs, but low-bit activation quantization remains difficult because activation outliers induce large quantization error.<n>We propose a lightweight post-training rotation calibration method for LLM activation quantization.
• Score: 14.352305549429289
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are costly to deploy due to their large memory footprint and high inference cost. Weight-activation quantization can reduce these costs, but low-bit activation quantization remains difficult because activation outliers induce large quantization error. Recent rotation-based methods address this by applying orthogonal transformations that redistribute activation magnitude across dimensions, but existing approaches either require expensive end-to-end rotation training or rely on stored activation corpora, introducing significant compute or storage overhead. We propose a lightweight post-training rotation calibration method for LLM activation quantization. Our method learns orthogonal rotations that align normalized activations with the corners of an inscribed hypercube, encouraging activation energy to be distributed more evenly across dimensions. This objective admits an efficient closed-form update via the orthogonal Procrustes problem, avoiding gradient-based optimization over the orthogonal group. We further introduce an online calibration procedure that updates rotations as calibration samples are processed, eliminating the need to store activations on disk and allowing rotations to adapt to quantized activation distributions during calibration. Experiments on Llama-2 and Llama-3 models from 3B to 70B parameters show that our method achieves competitive or improved performance across perplexity benchmarks and common sense reasoning tasks while avoiding both costly end-to-end training and large offline activation storage.

Related papers

• TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization [15.401450705732232]
  Ternarization has emerged as a promising compression technique.<n>Existing methods struggle with heavy-tailed activation distributions.<n>We propose TWLA, a framework that achieves 1.58-bit weight compression and 4-bit activation quantization.
  arXiv  Detail & Related papers  (2026-06-11T08:37:20Z)
• ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation [32.6734400537211]
  Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large Language Models (LLMs)<n>Layer-wise transformation methods emerged, achieving superior accuracy through localized adaptation.<n>However, layer-wise methods cannot fuse activation rotation matrices into weights, requiring online computations and causing significant overhead.<n>We propose ReSpinQuant, a quantization framework that resolves such overhead by leveraging offline activation rotation fusion and matching basis.
  arXiv  Detail & Related papers  (2026-04-13T07:00:26Z)
• FreeAct: Freeing Activations for LLM Quantization [89.97086263978058]
  Quantization is pivotal for mitigating the significant memory and computational overhead of Large Language Models.<n>FreeAct is a novel quantization framework that relaxes the static one-to-one constraint to accommodate dynamic activation disparities.<n>Experiments across dLLMs and MLLMs demonstrate that FreeAct significantly outperforms baselines, up to 5.3% performance improvement.
  arXiv  Detail & Related papers  (2026-03-02T12:02:17Z)
• Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantization [56.5199302532159]
  We propose an Activation-guided Structured Regularization framework to suppress the negative effects of outliers.<n>Astro actively reconstructs intrinsically robust weights, aggressively suppressing weight outliers corresponding to high-magnitude activations.<n>Astro achieves highly competitive performance; notably, on LLaMA-2-7B, it achieves better performance than complex learning-based rotation methods with almost 1/3 of the quantization time.
  arXiv  Detail & Related papers  (2026-02-07T15:50:18Z)
• Enhancing Post-Training Quantization via Future Activation Awareness [84.76726857601753]
  Post-training quantization (PTQ) is a widely used method to compress large language models (LLMs) without fine-tuning.<n>We propose Future-Aware Quantization (FAQ), which leverages future-layer activations to guide quantization.<n>FAQ consistently outperforms prior methods with negligible extra cost, requiring no backward passes, data reconstruction, or tuning.
  arXiv  Detail & Related papers  (2026-01-28T12:03:30Z)
• STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization [21.93314755695813]
  Quantization is the key method for reducing inference latency, power and memory footprint of generative AI models.<n>We propose textitSequence Transformation and Mixed Precision (STaMP) quantization.
  arXiv  Detail & Related papers  (2025-10-30T17:53:42Z)
• BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models [25.783531928577233]
  BASE-Q is a simple yet powerful approach that combines bias correction and asymmetric scaling to reduce rounding and clipping errors.<n>Experiments demonstrate the effectiveness of BASE-Q, narrowing the accuracy gap to full-precision models by 50.5%, 42.9%, and 29.2% compared to QuaRot, SpinQuant, and OSTQuant, respectively.
  arXiv  Detail & Related papers  (2025-05-26T14:22:21Z)
• RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models [53.571195477043496]
  We propose an algorithm named Rotated Straight-Through-Estimator (RoSTE)<n>RoSTE combines quantization-aware supervised fine-tuning (QA-SFT) with an adaptive rotation strategy to reduce activation outliers.<n>Our findings reveal that the prediction error is directly proportional to the quantization error of the converged weights, which can be effectively managed through an optimized rotation configuration.
  arXiv  Detail & Related papers  (2025-02-13T06:44:33Z)
• FlatQuant: Flatness Matters for LLM Quantization [58.28221892035609]
  We propose FlatQuant, a new post-training quantization approach that enhances the flatness of weights and activations.<n>Our approach identifies optimal affine transformations for each linear layer, calibrated in hours via a lightweight objective.<n>It achieves less than 1% accuracy drop for W4A4 quantization on the LLaMA-3-70B model, surpassing SpinQuant by 7.5%.
  arXiv  Detail & Related papers  (2024-10-12T08:10:28Z)
• Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference [54.2589824716527]
  Large language models incur substantial computation and memory movement costs due to their large scale. Existing approaches separate outliers and normal values into two matrices or migrate outliers from activations to weights, suffering from high latency or accuracy degradation. We propose Rotated Smooth (RRS), a plug-and-play activation smoother for quantization, consisting of Smooth and Rotation operation. The proposed method outperforms the state-of-the-art method in the LLaMA and Qwen families and improves WikiText-2 perplexity from 57.33 to 6.66 for INT4 inference.
  arXiv  Detail & Related papers  (2024-09-30T14:59:22Z)
• CBQ: Cross-Block Quantization for Large Language Models [66.82132832702895]
  Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs.<n>We propose CBQ, a cross-block reconstruction-based PTQ method for LLMs.<n> CBQ employs a cross-block dependency using a reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation.
  arXiv  Detail & Related papers  (2023-12-13T07:56:27Z)
• Efficient Adaptive Activation Rounding for Post-Training Quantization [18.174377380203033]
  Post-training quantization is convenience in deploying quantized neural networks. Previous research has demonstrated suboptimal nature when applied to weight quantization. We propose AQuant framework to reduce output error by adjusting rounding schemes of activations.
  arXiv  Detail & Related papers  (2022-08-25T09:02:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.