ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation

• URL: http://arxiv.org/abs/2604.11080v1
• Date: Mon, 13 Apr 2026 07:00:26 GMT
• Title: ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation
• Abstract summary: Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large Language Models (LLMs)<n>Layer-wise transformation methods emerged, achieving superior accuracy through localized adaptation.<n>However, layer-wise methods cannot fuse activation rotation matrices into weights, requiring online computations and causing significant overhead.<n>We propose ReSpinQuant, a quantization framework that resolves such overhead by leveraging offline activation rotation fusion and matching basis.
• Score: 32.6734400537211
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Rotation-based Post-Training Quantization (PTQ) has emerged as a promising solution for mitigating activation outliers in the quantization of Large Language Models (LLMs). Global rotation methods achieve inference efficiency by fusing activation rotations into attention and FFN blocks, but suffer from limited expressivity as they are constrained to use a single learnable rotation matrix across all layers. To tackle this, layer-wise transformation methods emerged, achieving superior accuracy through localized adaptation. However, layer-wise methods cannot fuse activation rotation matrices into weights, requiring online computations and causing significant overhead. In this paper, we propose ReSpinQuant, a quantization framework that resolves such overhead by leveraging offline activation rotation fusion and matching basis using efficient residual subspace rotation. This design reconciles the high expressivity of layer-wise adaptation with only negligible inference overhead. Extensive experiments on W4A4 and W3A3 quantization demonstrate that ReSpinQuant achieves state-of-the-art performance, outperforming global rotation methods and matching the accuracy of computationally expensive layer-wise methods with minimal overhead.

Related papers

• GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference [0.7122535777902205]
  GyRot is a low-bit quantization framework and hardware accelerator.<n>It achieves state-of-the-art 4-bit accuracy across LLaMA-family models.<n>It delivers up to 3.4x speedup and 3.6x energy efficiency over baseline LLM accelerators.
  arXiv  Detail & Related papers  (2026-07-30T05:26:20Z)
• TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization [15.401450705732232]
  Ternarization has emerged as a promising compression technique.<n>Existing methods struggle with heavy-tailed activation distributions.<n>We propose TWLA, a framework that achieves 1.58-bit weight compression and 4-bit activation quantization.
  arXiv  Detail & Related papers  (2026-06-11T08:37:20Z)
• ConQuR: Corner Aligned Activation Quantization via Optimized Rotations for LLMs [14.352305549429289]
  Large language models (LLMs) are costly to deploy due to their large memory footprint and high inference cost.<n>Weight-activation quantization can reduce these costs, but low-bit activation quantization remains difficult because activation outliers induce large quantization error.<n>We propose a lightweight post-training rotation calibration method for LLM activation quantization.
  arXiv  Detail & Related papers  (2026-05-11T16:23:10Z)
• RUQuant: Towards Refining Uniform Quantization for Large Language Models [17.258420059228808]
  Post-training quantization (PTQ) has emerged as a practical solution by compressing models without requiring retraining.<n>Existing methods often suffer from substantial accuracy degradation due to the non-uniform nature of activation distributions.<n>In this work, we revisit the activation quantization problem from a theoretical perspective grounded in the Lloyd-Max optimality conditions.
  arXiv  Detail & Related papers  (2026-04-05T08:04:39Z)
• SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization [7.372706701787234]
  Post-training quantization (PTQ) has emerged as a prevailing technique for deploying large language models efficiently.<n>We propose SERQ, a saliency-aware error reconstruction method for low-bit LLM inference that employs a single low-rank compensation matrix.
  arXiv  Detail & Related papers  (2026-03-09T10:04:12Z)
• MirrorLA: Reflecting Feature Map for Vision Linear Attention [49.41670925034762]
  Linear attention significantly reduces the computational complexity of Transformers from quadratic to linear, yet it consistently lags behind softmax-based attention in performance.<n>We propose MirrorLA, a geometric framework that substitutes passive truncation with active reorientation.<n>MirrorLA achieves state-of-the-art performance across standard benchmarks, demonstrating that strictly linear efficiency can be achieved without compromising representational fidelity.
  arXiv  Detail & Related papers  (2026-02-04T09:14:09Z)
• Parallel Diffusion Solver via Residual Dirichlet Policy Optimization [88.7827307535107]
  Diffusion models (DMs) have achieved state-of-the-art generative performance but suffer from high sampling latency due to their sequential denoising nature.<n>Existing solver-based acceleration methods often face significant image quality degradation under a low-dimensional budget.<n>We propose the Ensemble Parallel Direction solver (dubbed as EPD-EPr), a novel ODE solver that mitigates these errors by incorporating multiple gradient parallel evaluations in each step.
  arXiv  Detail & Related papers  (2025-12-28T05:48:55Z)
• BASE-Q: Bias and Asymmetric Scaling Enhanced Rotational Quantization for Large Language Models [25.783531928577233]
  BASE-Q is a simple yet powerful approach that combines bias correction and asymmetric scaling to reduce rounding and clipping errors.<n>Experiments demonstrate the effectiveness of BASE-Q, narrowing the accuracy gap to full-precision models by 50.5%, 42.9%, and 29.2% compared to QuaRot, SpinQuant, and OSTQuant, respectively.
  arXiv  Detail & Related papers  (2025-05-26T14:22:21Z)
• Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free [5.074812070492738]
  We introduce a novel, training-free approach to construct an improved rotation matrix, addressing the limitations of current methods.<n>Key contributions include leveraging the Walsh-Hadamard transform with sequency ordering, which clusters similar frequency components to reduce quantization error.<n>Our method demonstrates robust performance on reasoning tasks and Perplexity (PPL) score on WikiText-2.
  arXiv  Detail & Related papers  (2025-05-02T11:51:29Z)
• R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference [77.47238561728459]
  R-Sparse is a training-free activation sparsity approach capable of achieving high sparsity levels in advanced LLMs.<n> Experiments on Llama-2/3 and Mistral models across ten diverse tasks demonstrate that R-Sparse achieves comparable performance at 50% model-level sparsity.
  arXiv  Detail & Related papers  (2025-04-28T03:30:32Z)
• RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models [53.571195477043496]
  We propose an algorithm named Rotated Straight-Through-Estimator (RoSTE)<n>RoSTE combines quantization-aware supervised fine-tuning (QA-SFT) with an adaptive rotation strategy to reduce activation outliers.<n>Our findings reveal that the prediction error is directly proportional to the quantization error of the converged weights, which can be effectively managed through an optimized rotation configuration.
  arXiv  Detail & Related papers  (2025-02-13T06:44:33Z)
• Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference [54.2589824716527]
  Large language models incur substantial computation and memory movement costs due to their large scale. Existing approaches separate outliers and normal values into two matrices or migrate outliers from activations to weights, suffering from high latency or accuracy degradation. We propose Rotated Smooth (RRS), a plug-and-play activation smoother for quantization, consisting of Smooth and Rotation operation. The proposed method outperforms the state-of-the-art method in the LLaMA and Qwen families and improves WikiText-2 perplexity from 57.33 to 6.66 for INT4 inference.
  arXiv  Detail & Related papers  (2024-09-30T14:59:22Z)
• AffineQuant: Affine Transformation Quantization for Large Language Models [58.45460102764]
  Post-Training Quantization (PTQ) has emerged as a subject of considerable interest due to its compression efficiency and cost-effectiveness in the context of training. Existing PTQ methods for Large-scale Language Models (LLMs) limit the optimization scope to scaling transformations between pre- and post-quantization weights. In this paper, we advocate for the direct optimization using equivalent Affine transformations in PTQ (AffineQuant)
  arXiv  Detail & Related papers  (2024-03-19T08:40:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.