Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches

• URL: http://arxiv.org/abs/2607.20538v1
• Date: Fri, 10 Jul 2026 17:18:25 GMT
• Title: Codec-Gauge: Learning Compression-Friendly Gauges for Transformer KV Caches
• Abstract summary: Long-context Transformer inference increasingly relies on KV-cache compression or quantization.<n>We introduce Codec-Gauge, a post-training cache-coordinate layer that learns small channel transforms around existing compression and quantization backends.
• Score: 25.55174329283001
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-context Transformer inference increasingly relies on KV-cache compression or quantization. Prior rotation and transform-coding results suggest that the channel basis of each key/value vector affects how faithfully a fixed backend preserves model behavior. We introduce Codec-Gauge, a post-training cache-coordinate layer that learns small orthogonal channel transforms around existing compression and quantization backends. Its frequency-distribution objective combines a token-channel DCT spectral-centroid loss with a smooth rate proxy to concentrate KV energy in low-frequency codec-facing layouts. We evaluate actual compression and decompression using measured bytes and rolling compressed-history scoring. Across six models at $3$, $4$, and $6$ bits/value, learned gauges reduce zfp KL divergence by $44.0\%$ on average relative to raw coordinates and outperform random, Hadamard, DCT, and PCA/KLT controls. The same gauges improve quality preservation for block-uniform and KIVI-style quantization. Experiments on a 27B model and long-context task prompts reproduce the quality trend, while serial storage and timing measurements validate the implemented compressed-cache paths. These results establish cache-coordinate geometry as a practical post-training variable for improving compression fidelity without changing model weights, attention semantics, or backend coding rules.

Related papers

• KV Cache Compression Through the Lens of Transform Coding [9.884224071643606]
  We introduce Attention-Aware Transform Coding (AATC), which allocates bits over a calibration set to minimize attention-aware distortion.<n>Our method achieves near-lossless accuracy at approximately $5.8times$ compression, whereas each baseline degrades in at least some settings.
  arXiv  Detail & Related papers  (2026-08-14T11:08:01Z)
• DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression [0.0]
  DepthWeave-KV is a token-adaptive cache compression method that factorizes key and value states across neighboring transformer layers.<n> DepthWeave-KV achieves near-full-output task quality with substantially lower memory use.
  arXiv  Detail & Related papers  (2026-07-07T17:29:01Z)
• End-to-End Context Compression at Scale [81.70601323130997]
  Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.<n>Recent techniques to compress the KV cache fall short: they either degrade model quality substantially or require considerable time and compute to compress a single long prompt.<n>Existing approaches are not competitive with KV cache compression on the accuracy-efficiency frontier.
  arXiv  Detail & Related papers  (2026-06-08T15:43:16Z)
• OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization [13.284869342523095]
  Key-value (KV) dominates memory bandwidth and footprint in autoregressive inference.<n>Recent rotation-preconditioned codecs (TurboQuant, PolarQuant) show that a structured random rotation followed by a per-coordinate quantizer matched to an analytically tractable marginal is a near-optimal recipe for KV compression.<n>OCTOPUS advances this paradigm through joint quantization of rotated coordinate triplets.
  arXiv  Detail & Related papers  (2026-05-20T14:19:51Z)
• EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction [55.026048429595384]
  EchoKV is a flexible KV cache compression scheme that enables on-demand transitions between standard and compressed inference.<n>We introduce a two-stage fine-tuning strategy that allows for rapid, low-cost training.
  arXiv  Detail & Related papers  (2026-03-24T07:58:42Z)
• CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing [54.34080239841088]
  CommonKV is a training-free method for cross-layer KV cache compression through adjacent parameters sharing.<n>We show that the proposed method consistently outperforms existing low-rank and cross-layer approaches at various compression ratios.
  arXiv  Detail & Related papers  (2025-08-22T06:55:45Z)
• KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding [72.12756830560217]
  Large language models (LLMs) based on Transformer Decoders have become the preferred choice for conversational generative AI.<n>Despite the overall superiority of the Decoder architecture, the gradually increasing Key-Value cache during inference has emerged as a primary efficiency bottleneck.<n>By down-sampling the Key-Value vector dimensions into a latent space, we can significantly reduce the KV Cache footprint and improve inference speed.
  arXiv  Detail & Related papers  (2025-07-15T12:52:12Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy [59.1298692559785]
  Key-Value ( KV) cache is crucial component in serving transformer-based autoregressive large language models (LLMs) Existing approaches to mitigate this issue include: (1) efficient attention variants integrated in upcycling stages; (2) KV cache compression at test time; and (3) KV cache compression at test time. We propose a low-rank approximation of KV weight matrices, allowing plug-in integration with existing transformer-based LLMs without model retraining. Our method is designed to function without model tuning in upcycling stages or task-specific profiling in test stages.
  arXiv  Detail & Related papers  (2024-10-04T03:10:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.