Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression

• URL: http://arxiv.org/abs/2605.27646v1
• Date: Tue, 26 May 2026 20:09:17 GMT
• Title: Hurwitz Quaternion Multiplicative Quantization for KV Cache Compression
• Abstract summary: HQMQ treats each 4-element chunk of K or V as a quaternion and quantizes its unit direction to the emphproduct $q_p cdot q_s$.<n> HQMQ delivers up to $5.05times$ KV compression, shrinking a Llama-3-70B 128k-context cache from 43 GB to 8.5 GB.
• Score: 19.924689669010117
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose \textbf{Hurwitz Quaternion Multiplicative Quantization (HQMQ)}, a \textbf{calibration-free} method for KV cache compression of large language models. HQMQ treats each 4-element chunk of K or V as a quaternion and quantizes its unit direction to the \emph{product} $q_p \cdot q_s$, where $q_p$ ranges over the 24-element Hurwitz group $2T$ (the 24 vertices of the 24-cell on $S^3$, pairwise angle $60^\circ$) and $q_s$ ranges over a per-(layer, head) secondary codebook of $S$ \emph{random} unit quaternions. The multiplicative composition yields $24S$ effective codewords at $S$ stored parameters; random initialization suffices because left-multiplication is an $S^3$ isometry, so seeded codebooks vary in end-task ppl by $<1.5\%$. A per-batch median-multiplier outlier extraction step ($C{=}3$, no calibration) handles modern outlier-heavy architectures. We evaluate on five modern open models: Mistral-7B (dense MHA), Llama-3-8B and Qwen2.5-7B and Qwen3-8B (dense GQA), and gpt-oss-20b (sparse MoE). On Mistral-7B and Qwen3-8B, HQMQ matches fp16 within $0.02$--$0.03$ ppl points at $\sim$5 bits. On Qwen2.5-7B and Qwen3-8B, where naive int4 collapses to $10^4{+}$ ppl, HQMQ + Med3$\times$ recovers fp16 quality within $0.02$--$0.10$ ppl points at $\sim$5 bits. HQMQ Pareto-dominates naive int by $3$--$1900\times$ at matched bits across all five models, and downstream zero-shot accuracy matches fp16 at $3.79$ bits on Mistral. Against the strongest calibrated KV-quantization baseline, HQMQ at $3.79$ bits matches KIVI-4 ($\sim 4.5$ bits) within ${\sim}1$ pt on CoQA, $0.6$ pts on TruthfulQA, and $2.3$ pts on GSM8K, at $16\%$ fewer bits and without a calibration pass. At the storage level, HQMQ delivers up to $5.05\times$ KV compression, shrinking a Llama-3-70B 128k-context cache from 43 GB to 8.5 GB.

Related papers

• The Condition-Number Barrier in Sparse Least Squares [77.64108812086542]
  In [AS21], Axiotis and Sviridenko conjectured that the linear dependence on the restricted condition number in convex optimization cannot be improved by a sparse-time algorithm.<n>We establish their conjectured lower bound for least-squares objectives, conditional on the randomized exact-volume Small-Set Expansion Hypothesis.
  arXiv  Detail & Related papers  (2026-08-03T17:57:01Z)
• ExTernD: Expanded-Rank Ternary Decomposition Ternary LLM PTQ with Accuracy Approaching Any Quantization Level [0.0]
  ExTernD is a post-training factorization of each LLM weight matrix $A in mathbbRm times n$ into $A approx B mathrmdiag(D) C$ with ternary factors $B in -1,0,+1m times k$, $C in -1,0,+1k times n$ and a real scale vector $D in mathbbRk$.
  arXiv  Detail & Related papers  (2026-07-15T07:04:32Z)
• An End-to-End Hybrid Quantum--Classical Sampling Workflow for Discrete Markov Random Fields: A Reproducible Case Study [0.0]
  We study amplitude-encoded i.i.d. sampling for small discrete Markov random fields (MRFs)<n>This removes quantum exponential speedup but allows a clean comparison against classical MCMC based on independent circuit samples.<n>A matched-budget VQC vs. MPS comparison shows VQC fidelities fall far below MPS: $(F_mathrmVQC, F_mathrmMPS) = (0.31, 0.96), (0.21, 0.96), (0.17, 0.88)$ at compressions $10.7times$, $34.1times$, and $11
  arXiv  Detail & Related papers  (2026-07-10T18:33:14Z)
• An IQP Born Machine for Calorimeter Image Generation at 64 Qubits with Compiled-IQP Deployment [0.0]
  We train an instantaneous quantum-time (IQP) Born machine on real high-energy-physics calorimeter shower images at $64$ qubits.<n>We compile the trained model into a single sampling-hard IQP circuit for quantum deployment.
  arXiv  Detail & Related papers  (2026-05-26T22:18:55Z)
• When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon [0.0]
  KV-cache quantization is framed as a quality--latency trade-off.<n>We show it is emphinverted on Apple Silicon's unified memory.
  arXiv  Detail & Related papers  (2026-05-07T05:44:39Z)
• ZenBrain: A Neuroscience-Inspired 7-Layer Memory Architecture for Autonomous AI Systems [51.56484100374058]
  On LongMemEval-500, ZenBrain matches a long-context oracle's binary-judge accuracy to within 4.5 pp.<n> ZenBrain is a 7-layer neuroscience-inspired memory architecture.
  arXiv  Detail & Related papers  (2026-04-26T20:39:19Z)
• High-Girth Regular Quantum LDPC Codes from Affine-Coset Structures [0.0]
  We construct a quantum low-density parity-check code family from a length-$512$ Calderbank-Shor--Steane base matrix pair.<n>The base pair is permutation-equivalent to the known SPC(3) product CSS code.
  arXiv  Detail & Related papers  (2026-04-22T17:59:08Z)
• Latent Phase-Shift Rollback: Inference-Time Error Correction via Residual Stream Monitoring and KV-Cache Steering [4.032680910442999]
  Large language models commit unrecoverable reasoning errors mid-generation.<n>We introduce $textbfLatent Phase-Shift Rollback$ (LPSR)<n>At each generation step, we monitor the residual stream at a critical layer lcrit, detect abrupt directional reversals (phase shifts) via a cosine-similarity $+$ entropy dual gate.<n>No fine-tuning, gradient computation, or additional forward passes are required.
  arXiv  Detail & Related papers  (2026-04-20T17:53:33Z)
• IsoQuant: Hardware-Aligned SO(4) Isoclinic Rotations for LLM KV Cache Compression [0.4496256885343706]
  We propose a blockwise rotation framework based on quaternion algebra and the isoclinic decomposition of $SO(4)$.<n>IsoQuant achieves mean kernel-level speedups of $4.5times$--$4.7times$ over RotorQuant with peak speedups above $6times$.
  arXiv  Detail & Related papers  (2026-03-30T13:37:45Z)
• Approximating the operator norm of local Hamiltonians via few quantum states [53.16156504455106]
  Consider a Hermitian operator $A$ acting on a complex Hilbert space of $2n$.<n>We show that when $A$ has small degree in the Pauli expansion, or in other words, $A$ is a local $n$-qubit Hamiltonian.<n>We show that whenever $A$ is $d$-local, textiti.e., $deg(A)le d$, we have the following discretization-type inequality.
  arXiv  Detail & Related papers  (2025-09-15T14:26:11Z)
• Algebraic Aspects of Boundaries in the Kitaev Quantum Double Model [77.34726150561087]
  We provide a systematic treatment of boundaries based on subgroups $Ksubseteq G$ with the Kitaev quantum double $D(G)$ model in the bulk. The boundary sites are representations of a $*$-subalgebra $Xisubseteq D(G)$ and we explicate its structure as a strong $*$-quasi-Hopf algebra. As an application of our treatment, we study patches with boundaries based on $K=G$ horizontally and $K=e$ vertically and show how these could be used in a quantum computer
  arXiv  Detail & Related papers  (2022-08-12T15:05:07Z)
• Quantum double aspects of surface code models [77.34726150561087]
  We revisit the Kitaev model for fault tolerant quantum computing on a square lattice with underlying quantum double $D(G)$ symmetry. We show how our constructions generalise to $D(H)$ models based on a finite-dimensional Hopf algebra $H$.
  arXiv  Detail & Related papers  (2021-06-25T17:03:38Z)
• Deep Learning Meets Projective Clustering [66.726500395069]
  A common approach for compressing NLP networks is to encode the embedding layer as a matrix $AinmathbbRntimes d$. Inspired by emphprojective clustering from computational geometry, we suggest replacing this subspace by a set of $k$ subspaces.
  arXiv  Detail & Related papers  (2020-10-08T22:47:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.