ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection

• URL: http://arxiv.org/abs/2602.08343v1
• Date: Mon, 09 Feb 2026 07:28:55 GMT
• Title: ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection
• Abstract summary: We propose a training-free scorer that ranks tokens by Euclidean distance to the key centroid.<n>ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression.<n> WindowedManifoldKV restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff.
• Score: 8.362927764080203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-context inference is constrained by KV-cache memory, which grows linearly with sequence length; KV-cache compression therefore hinges on reliably selecting which past tokens to retain. Most geometry-based eviction methods score keys by cosine similarity to a global centroid, but cosine is scale-invariant and can discard magnitude cues that distinguish semantically salient tokens. We propose ManifoldKV, a training-free scorer that ranks tokens by Euclidean distance to the key centroid, capturing both angular and radial deviations. On the RULER benchmark, ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression; matching the best geometric baseline while improving robustness in two regimes where cosine scoring fails. First, on multi-key retrieval, ManifoldKV reduces directional collisions, achieving 92.4% vs KeyDiff's 77.0% (+15.4 points) on 3-key NIAH at 50% compression. Second, to address dilution and performance collapse of global centroids at 64K context, we introduce WindowedManifoldKV, which restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff. The method requires only 3 lines of code and works across 4 architectures without tuning.

Related papers

• FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference [0.0]
  FreqDepthKV is an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals.<n>A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits.<n>Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets.
  arXiv  Detail & Related papers  (2026-07-07T17:26:28Z)
• Output-Space Allocation Costs for Calibration-Guided LLM Compression: An Empirical Study [19.413175066072302]
  Training-free compression methods for large language models (LLMs) often use calibration data to guide compression decisions.<n>ROCKET, a recent method combining sparse-dictionary factorization with multi-choice knapsack problem (MCKP) allocation, derives its per-layer factorization from an output reconstruction objective.<n>We investigate whether aligning the allocation cost with the output-space objective improves compressed model fidelity.
  arXiv  Detail & Related papers  (2026-06-26T07:15:17Z)
• NestedKV: Nested Memory Routing for Long-Context KV Cache Compression [35.62789874560166]
  NestedKV is a key-only KV cache compression method inspired by the Continuum Memory System in Nested Learning.<n>It scores tokens by multi-time-scale cosine anomaly, and combines the resulting rankings with a training-free outer learner.
  arXiv  Detail & Related papers  (2026-05-26T08:14:39Z)
• KVSculpt: KV Cache Compression as Distillation [7.085426079187912]
  KV cache compression is critical for efficient long-context LLM inference.<n>Existing methods range from pure eviction -- selecting which KV pairs to keep -- to merging, which combines similar pairs into fewer ones.<n>We propose KVSculpt, which moves to the other end of this spectrum.<n>Keys are optimized via L-BFGS and values are solved in closed form via least squares.
  arXiv  Detail & Related papers  (2026-03-29T19:14:25Z)
• LOOKAT: Lookup-Optimized Key-Attention for Memory-Efficient Transformers [0.0]
  Current quantization methods compress storage but fail to reduce bandwidth as attention calculation requires dequantizing keys from INT4/INT8 to FP16 before use.<n>We propose LOOKAT, which applies product quantization and asymmetric distance computation to transformer architecture by decomposing key vectors into subspaces.
  arXiv  Detail & Related papers  (2026-01-15T07:54:07Z)
• CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing [54.34080239841088]
  CommonKV is a training-free method for cross-layer KV cache compression through adjacent parameters sharing.<n>We show that the proposed method consistently outperforms existing low-rank and cross-layer approaches at various compression ratios.
  arXiv  Detail & Related papers  (2025-08-22T06:55:45Z)
• Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs [28.554617623874176]
  We show a fundamental yet previously overlooked asymmetry in KV caches.<n>Key-value asymmetry reveals a critical limitation in existing compression methods.<n>We propose a training-free compression framework (AsymKV) that combines homogeneity-based key merging with a mathematically proven lossless value compression.
  arXiv  Detail & Related papers  (2025-06-04T16:10:44Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search [61.4807238517108]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by enabling step-by-step problem-solving.<n>CoT's extension to Long-CoT introduces substantial computational overhead due to increased token length.<n>We propose R1-Compress, a two-stage chunk-level compression framework that preserves both local information and coherence.
  arXiv  Detail & Related papers  (2025-05-22T16:06:59Z)
• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference [61.412894960600205]
  Large Language Models (LLMs) require significant GPU memory when processing long texts.<n>ChunkKV reimagines KV cache compression by treating semantic chunks as basic compression units.<n>Result: ChunkKV outperforms state-of-the-art methods by up to 8.7% in precision.
  arXiv  Detail & Related papers  (2025-02-01T03:49:47Z)
• EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance [44.14919492126948]
  As memory overhead becomes a significant concern, efficient compression of KV cache has gained increasing attention.<n>We propose EMS to overcome these limitations, while achieving better KV cache compression under extreme compression ratios.<n> EMS consistently achieves the lowest perplexity, improves scores by over 1.28 points across four LLMs on LongBench under a 256 cache budget, and preserves 95% retrieval accuracy with a cache budget less than 2% of the context length in the Needle-in-a-Haystack task.
  arXiv  Detail & Related papers  (2024-12-11T16:35:13Z)
• KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing [58.29726147780976]
  We propose a plug-and-play method called textit KVSharer, which shares the KV cache between layers to achieve layer-wise compression. Experiments show that textit KVSharer can reduce KV cache computation by 30%, thereby lowering memory consumption. We verify that textit KVSharer is compatible with existing intra-layer KV cache compression methods, and combining both can further save memory.
  arXiv  Detail & Related papers  (2024-10-24T08:06:41Z)
• KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head [0.8158530638728501]
  We introduce KV-Compress, a novel compression method that evicts contiguous KV blocks within a PagedAttention framework. Our method achieves state-of-the-art performance on LongBench for both Mistral-7B-Instruct-v0.2 and Llama-3.1-8B-Instruct while lowering the total number of compressed KVs by 4x. Evaluations on Llama-3.1-8B-Instruct and Llama-3.1-70B-Instruct-FP8 achieve compression rates up to 8x with negligible impact on performance, and up to 64x while retaining over 90% of full-cache performance
  arXiv  Detail & Related papers  (2024-09-30T19:09:13Z)
• LoCoCo: Dropping In Convolutions for Long Context Compression [77.26610232994508]
  This paper presents a novel approach, Dropping In Convolutions for Long Context Compression (LoCoCo) LoCoCo employs only a fixed-size Key-Value ( KV) cache, and can enhance efficiency in both inference and fine-tuning stages.
  arXiv  Detail & Related papers  (2024-06-08T01:35:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.