OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization

• URL: http://arxiv.org/abs/2605.21226v1
• Date: Wed, 20 May 2026 14:19:51 GMT
• Title: OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
• Abstract summary: Key-value (KV) dominates memory bandwidth and footprint in autoregressive inference.<n>Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression.<n>OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets.
• Score: 13.284869342523095
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The key-value (KV) cache dominates memory bandwidth and footprint in long-context autoregressive inference. Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate scalar quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression. OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets. Each triplet's direction is mapped to a square via an octahedral parameterization, and the two resulting coordinates and the triplet norm are Lloyd-Max quantized against implementation-matched marginals. Optimizing the per-triplet squared error gives a strictly non-uniform bit allocation depending only on the total dimensionality of the keys. We find the finite-dimensional quality optimum with sweeps to be constant on every real decoder we test. The codec is data-oblivious, online, and deterministic given a seed. Across text, video, and audio, OCTOPUS matches or beats every prior rotation codec at every reported bit width and metric, with a lead that grows as bits drop for extreme compression. Furthermore, a fused Triton implementation reconstructs keys on the fly without materializing the uncompressed key, so the codec adds no decode-time bandwidth or latency over the existing dequantization. Project Page: https://octopus-quant.github.io/

Related papers

• Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches [25.55174329283001]
  Long-context Transformer inference increasingly relies on KV-cache compression or quantization.<n>We introduce Codec-Gauge, a post-training cache-coordinate layer that learns small channel transforms around existing compression and quantization backends.
  arXiv  Detail & Related papers  (2026-07-10T17:18:25Z)
• End-to-End Context Compression at Scale [81.70601323130997]
  Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.<n>Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.<n>Existing approaches are not competitive with KV cache compression on the accuracy-efficiency frontier.
  arXiv  Detail & Related papers  (2026-06-08T15:43:16Z)
• Beyond Encoder Accumulation: Measuring Encoder Roles in Multi-Encoder VLMs [50.3757247538081]
  We retrain and evaluate all 31 non-empty subsets of five common vision encoders under a unified pipeline.<n>We decompose each encoder's contribution into two axes, Capacity and Necessity.<n>Per-encoder pre-projector effective rank explains the residual score variation.
  arXiv  Detail & Related papers  (2026-06-02T16:46:42Z)
• CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting [53.50198425973128]
  We introduce CodecSplat, an ultracompact latent coding framework for feed-forward 3D Gaussian splatting.<n>CodecSplat achieves 23.56-26.36 dB and 24.76-27.05 dB with only 20.00-10777 PSNR with 3.37-12 and 3.37-12 respectively.
  arXiv  Detail & Related papers  (2026-05-25T08:17:35Z)
• OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond [50.440302567029654]
  Multi-modal intelligence has made the Key-Value cache a dominant memory bottleneck for efficient deployment.<n>In this work, we revisit the inherent limitations of the per-channel quantization paradigm.<n>We propose OScaR, an accurate and lightweight KV cache compression framework for X-LLMs.
  arXiv  Detail & Related papers  (2026-05-19T10:53:03Z)
• FibQuant: Universal Vector Quantization for Random-Access KV-Cache Compression [35.73320708128261]
  We introduce textscFibQuant, a universal fixed-rate vector quantizer.<n>The codebook combines Beta-quantile radii, Fibonacci,/,Roberts--Kronecker quasi-Max refinement.<n>On GPT-2 small KV caches, textscFibQuant traces a memory--fidelity frontier.
  arXiv  Detail & Related papers  (2026-05-12T03:45:53Z)
• MesonGS++: Post-training Compression of 3D Gaussian Splatting with Hyperparameter Searching [27.114461961869782]
  3D Gaussian Splatting (3DGS) achieves high-quality novel view with real-time rendering, but its storage cost remains prohibitive for practical deployment.<n>We propose MesonGS, a size-aware post-training for 3D Gaussian compression. On the side, MesonGS combines joint importance-based pruning, octree geometry, transformations, selective vector quantization for higher-degree spherical harmonics, and group-wise mixed-precision quantization with entropy coding.
  arXiv  Detail & Related papers  (2026-04-29T15:30:06Z)
• 3DTurboQuant: Training-Free Near-Optimal Quantization for 3D Reconstruction Models [2.0789144738685956]
  Existing method for compressing 3D reconstructors requires learning a data-dependent codebook through per-scene fine-tuning.<n>We show this is unnecessary. parameter vectors that dominate storage in these models fall in a dimension range where a single random rotation transforms any input into coordinates with a known Beta distribution.<n>We develop 3D, deriving (1) a dimension-dependent criterion that predicts which parameters can be quantized and at what bit-width before running any experiment, (2) norm-separation bounds connecting quantization MSE to rendering PSNR per scene, and (3) an entry-grouping strategy extending rotation-based quantization
  arXiv  Detail & Related papers  (2026-04-07T03:06:05Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• Quantize What Counts: More for Keys, Less for Values [63.51476878610841]
  Large Language Models (LLMs) suffer inference-time memory bottlenecks dominated by the attention Key-Value ( KV) cache.<n>This paper proposes two theorems that anchor mixed-precision KV quantization in the intrinsic geometry of Transformer models.
  arXiv  Detail & Related papers  (2025-02-20T22:24:27Z)
• Residual vector quantization for KV cache compression in large language model [2.3094645821058735]
  KV cache compression methods have mainly relied on scalar quantization techniques to reduce the memory requirements during decoding. In this work, we apply residual vector quantization, which has been widely used for high fidelity audio compression, to compress KV cache in large language models (LLM) We learn the codebook using exponential moving average and there are no other learnable parameters including the input and output projections normally used in a vector quantization set up.
  arXiv  Detail & Related papers  (2024-10-21T07:20:41Z)
• Let the Code LLM Edit Itself When You Edit the Code [50.46536185784169]
  underlinetextbfPositional textbfIntegrity textbfEncoding (PIE)<n>PIE reduces computational overhead by over 85% compared to the standard full recomputation approach.<n>Results demonstrate that PIE reduces computational overhead by over 85% compared to the standard full recomputation approach.
  arXiv  Detail & Related papers  (2024-07-03T14:34:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.