QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling

• URL: http://arxiv.org/abs/2605.26339v1
• Date: Mon, 25 May 2026 21:28:46 GMT
• Title: QAM-W: Joint 2D Codebook Quantization for LLM Weights via Hadamard Rotation and Activation-Aware Scaling
• Abstract summary: Scalar post-training quantizers discard pairwise coordinate structure within weight rows.<n>We introduce QAM-W (Quadrature Amplitude Modulation for Weights), a variant that recovers this structure.<n>A 3.5bpw variant is competitive on quantization-tolerant architectures.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Scalar post-training quantizers discard pairwise coordinate structure within weight rows. We introduce QAM-W (Quadrature Amplitude Modulation for Weights), a codec that recovers this structure: each row is L2-normalized, block-Hadamard rotated, paired into 2D coordinates, and quantized against a single Lloyd-Max codebook trained on the unit circular Gaussian, with activation-aware per-channel scaling. In a cross-model study spanning five LLMs from four families (1.1B--13B parameters) and eight quantized configurations, the activation-aware variant at $\approx 5.5$ bpw stays within $\pm 0.4\%$ of BF16 WikiText-2 perplexity on every model, matching the SmoothQuant W8A8 quality envelope at $32\%$ fewer weight bits. Joint 2D coding outperforms polar (amplitude $\times$ phase) coding by 2--15~pp $Δ$PPL at equal bitrate, and paired KL against BF16 tracks $Δ$PPL\% at Spearman $ρ= 0.99$ across 37 (method, model) rows, consistent with a monotone composite bound from codec distortion to KL divergence. A 3.5~bpw variant is competitive on quantization-tolerant architectures. At strict 4~bpw, the rotated-codebook frontier method QTIP outperforms QAM-W; the contribution is the quality-preserving 5--6~bpw band.

Related papers

• SoftWater: Class-Aware Rate Allocation for Softmax Quantization [5.213778368155993]
  Post-training quantization pipelines routinely leave the softmax output layer in high precision.<n>In small LLMs with modern vocabularies, the head holds 15--30% of all parameters, so a nominal 2-bit'' model with an fp16 head can store several times as many bits per weight.<n>We pose softmax-layer quantization as a rate-distortion problem under the KL divergence between the original and quantized output distributions.
  arXiv  Detail & Related papers  (2026-08-12T13:06:50Z)
• KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers [42.11461757540461]
  Post-training quantization of diffusion transformers (DiTs) to W4A4 severely degrades output quality.<n>We propose KroQuant, a PTQ method that applies a learned Kronecker-structured invertible transform to each 32-element block of the activation.
  arXiv  Detail & Related papers  (2026-07-23T15:52:20Z)
• SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inference [4.219937026460372]
  SharQ bridges activation sparsity and FP4 quantization through an online sparse--dense decomposition.<n>A sparse FP4 GEMM processes the backbone while a dense FP4 GEMM compensates for both mask-induced activation loss and sparse-path quantization error.
  arXiv  Detail & Related papers  (2026-06-25T04:19:04Z)
• HyperQuant: A Rate-Distortion-Optimal Quantization Pipeline for Large Language and Diffusion Models [0.0]
  HyperQuant is a unified post-training quantization pipeline for the weights and the KV cache of large language and diffusion transformers.<n>It outperforms the recent HIGGS scheme at every operating point from 3 to 5 bits per scalar (bps) on weights, and beats both TurboQuant and OCTOPUS on KV quantization down to 1.7 bps.<n>It compresses the linear weights 3.9x and the KV cache 3.79x at near-lossless quality.
  arXiv  Detail & Related papers  (2026-06-22T14:30:47Z)
• Breaking the bicycle frame: Coset-based quantum LDPC codes [65.96089068176079]
  Generalizing the construction of two-block group algebra (2BGA) codes, we introduce a family of two-block quantum LDPC codes.<n>We identify several new quantum LDPC codes, including weight-6 codes with parameters $[48,8,6]]$, $[96,8,10]]$, and $[224,12,16]]$.<n>Our codes, when decoded using BP-OSD, perform competitively with BB codes, achieving thresholds of $approx0.65%$ for the weight-6 family and $approx0.35%$ for the weight-8 family.
  arXiv  Detail & Related papers  (2026-06-15T20:20:57Z)
• Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models [50.16014098038291]
  Post-training quantization (PTQ) is critical for the efficient deployment of large language models (LLMs)<n>We propose SAGE-PTQ, a novel ultra-low-bit quantization framework for LLMs that minimizes hidden scaling cost.<n>On LLaMA-3B, SAGE-PTQ achieves 6.74 WikiText2 perplexity, compared to 55.8 for BiLLM, while using less than 50% of BiLLM's GPU memory.
  arXiv  Detail & Related papers  (2026-06-03T20:51:52Z)
• Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference [3.308743964406687]
  Two-bit weight quantization is attractive for memory-efficient LLM inference.<n>The standard W2 level set -2,-1,0,+1 often collapses under aggressive W2A4/KV4 settings.<n>We propose Qift, a fixed no-zero W2 level set for rotated W2A4/KV4 inference.
  arXiv  Detail & Related papers  (2026-06-01T19:40:32Z)
• Influence-Inspired Spectral Rotations for Extreme Low-Bit LLM Quantization [0.0]
  We apply the influence of a companion theory paper to extreme low-bit weight-only quantization.<n>We show that BBTspectral reduces wikitext-2 perplexity by 58% relative to vanilla autoround at W2A16.
  arXiv  Detail & Related papers  (2026-05-24T18:05:37Z)
• ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models [45.62029693245481]
  We introduce ActQuant, an action-guided mixed-precision PTQ framework.<n>ActQuant operates in two stages: (1) an inter-tensor bit allocator that assigns each weight matrix a single bitwidth based on how much it contributes to predicting the agent's actions.<n>We also introduce OmniModel, an agentic conversion pipeline that ports into a native C/C++ with efficient low-bit kernels.
  arXiv  Detail & Related papers  (2026-05-19T19:57:26Z)
• BWTA: Accurate and Efficient Binarized Transformer by Algorithm-Hardware Co-design [71.97035034203275]
  We analyze zero-point distortion in binarization and propose a Binary Weights & Ternary Activations (BWTA) quantization scheme.<n>For training, we propose Smooth Multi-Stage Quantization, combining a Levelwise Degradation Strategy and a MagnitudeAlignment Projection Factor.<n>Experiments show that BWTA approaches full-precision performance for Transformer-based models, with an average 3.5% drop on GLUE and less than 2% drop on five tasks.
  arXiv  Detail & Related papers  (2026-04-05T04:25:07Z)
• IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression [0.4496256885343706]
  We propose a blockwise rotation framework based on quaternion algebra and the isoclinic decomposition of $SO(4)$.<n>IsoQuant achieves mean kernel-level speedups of $4.5times$--$4.7times$ over RotorQuant with peak speedups above $6times$.
  arXiv  Detail & Related papers  (2026-03-30T13:37:45Z)
• Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizer [7.833559115428751]
  Rotate, Clip, and Partition (RCP) is a quantization-aware training (QAT) approach.<n>RCP integrates recent rotation techniques with a novel non-uniform weight quantizer design.<n>RCP can compress LLaMA-2-7B to W2A4KV4 with a loss of only 2.84 WikiText2 ppl and 5.29 times reduced memory footprint.
  arXiv  Detail & Related papers  (2025-02-17T08:12:34Z)
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models [61.474101404805545]
  Diffusion models can generate high-quality images, but as they scale, rising memory demands and higher latency pose deployment challenges.<n>We propose SVDQuant, a new 4-bit quantization paradigm to overcome this limitation.<n>We reduce the memory usage for the 12B FLUX.1 models by 3.5$times$, achieving 3.0$times$ speedup over the 4-bit weight-only quantization (W4A16) baseline.
  arXiv  Detail & Related papers  (2024-11-07T18:59:58Z)
• FlatQuant: Flatness Matters for LLM Quantization [58.28221892035609]
  We propose FlatQuant, a new post-training quantization approach that enhances the flatness of weights and activations.<n>Our approach identifies optimal affine transformations for each linear layer, calibrated in hours via a lightweight objective.<n>It achieves less than 1% accuracy drop for W4A4 quantization on the LLaMA-3-70B model, surpassing SpinQuant by 7.5%.
  arXiv  Detail & Related papers  (2024-10-12T08:10:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.