How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit

• URL: http://arxiv.org/abs/2607.11942v1
• Date: Sat, 11 Jul 2026 09:30:35 GMT
• Title: How Query Visibility Changes KV-Cache Compression Rankings: A Matched-Budget Audit
• Abstract summary: KV-cache compression methods are predominantly evaluated with the query appended to the context before compression.<n>We present a matched-budget audit of six published compression methods against three trivial baselines.
• Score: 5.366718741784969
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: KV-cache compression methods are predominantly evaluated with the query appended to the context before compression -- a query-aware protocol. Yet the economic case for a compressed KV cache is reuse: compress a document once, answer many future questions against it. In that deployment, compression must happen query-agnostic -- before any question is seen. We present a matched-budget audit of six published compression methods against three trivial baselines on three open 7-9B models (144,300 paired evaluations on RULER-8192; 40,800 on LongBench; 50,000-resample paired bootstrap throughout). Everything is held fixed -- model, compression ratio, instances, decoding -- except the scoring rule. Three findings. (1) Query visibility changes the rankings: under the agnostic protocol, of the five audited methods that share a common attention backend, only KeyDiff beats a best-of-3 trivial baseline consistently (31 of 36 cells), and the most widely deployed method, SnapKV, loses to "keep the start and the recent window" on average (-0.066). (2) The per-method drop between the two protocols is ordered consistently with how visible the question is to each method's scoring signal, legible in its source code: from Delta=+0.198 for SnapKV (the question sits inside its 64-token observation window) down to Delta=+0.011 for KeyDiff (its score contains no query term at all).

Related papers

• Data Diversity, Not Frequency Invariance: A Controlled and Self-Audited Study of Compression-Robust Deepfake Detection [0.26856688022781555]
  Frequency features and compression-invariant representation learning are widely assumed to be key to deepfake detection that survives video compression.<n>We test this with CAFRL - block-DCT and FFT-phase streams, compression-level-conditioned band attention, and adversarial (gradient-reversal) compression.<n>Under a pre-registered protocol with capacity- and augmentation-matched controls, a plain EfficientNet-B0 on multi-quality data beat CAFRL as specified at every compression level.
  arXiv  Detail & Related papers  (2026-08-26T12:05:10Z)
• KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models [5.349156411852564]
  KV-cache compression reduces long-context memory, but aggregate task scores reveal neither which correct executions fail nor why.<n>We present KVDiagnosis, a diagnostic dataset and benchmark with three contributions.
  arXiv  Detail & Related papers  (2026-08-10T10:38:05Z)
• Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression [18.435649248662944]
  We show that when retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling.<n>We train a compact to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference.
  arXiv  Detail & Related papers  (2026-08-05T07:58:53Z)
• Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching [7.336436436266022]
  We characterize the cost empirically on Anthropic's Sonnet 4.6 API.<n>We propose Cache-Aware Prompt Compression (CAPC)<n>CAPC is the cheapest strategy in 16/16 configurations on LongBench-v2.
  arXiv  Detail & Related papers  (2026-07-17T00:03:47Z)
• FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference [0.0]
  FreqDepthKV is an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals.<n>A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits.<n>Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets.
  arXiv  Detail & Related papers  (2026-07-07T17:26:28Z)
• Fixed RAG Compression Collapses Measured Reader Scaling [2.9089118242427627]
  We show fixed compression can raise average accuracy while hiding reader upgrades and reversing model rankings.<n>We release ragscale, a toolkit built on 177,000 row-level compression transitions to audit reader scaling with three readers in one day.
  arXiv  Detail & Related papers  (2026-06-20T00:04:31Z)
• Multi-Vector Index Compression in Any Modality [73.7330345057813]
  Late interaction has emerged as a dominant paradigm for information retrieval in text, images, visual documents, and videos.<n>We introduce four approaches for index compression: sequence resizing, memory tokens, hierarchical pooling, and a novel attention-guided clustering (AGC)<n>AGC uses an attention-guided mechanism to identify the most semantically salient regions of a document as cluster centroids and to weight token aggregation.
  arXiv  Detail & Related papers  (2026-02-24T18:57:33Z)
• Arbitrary Ratio Feature Compression via Next Token Prediction [52.10426317889982]
  Arbitrary Ratio Feature Compression (ARFC) framework supports any compression ratio with a single model.<n>ARC is an auto-regressive model that performs compression via next-gressive prediction.<n>MoS module refines the compressed tokens by utilizing multiple compression results.<n>ERGC is integrated into the training process to preserve semantic and structural relationships during compression.
  arXiv  Detail & Related papers  (2026-02-12T02:38:57Z)
• ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection [8.362927764080203]
  We propose a training-free scorer that ranks tokens by Euclidean distance to the key centroid.<n>ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression.<n> WindowedManifoldKV restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff.
  arXiv  Detail & Related papers  (2026-02-09T07:28:55Z)
• Are We Using the Right Benchmark: An Evaluation Framework for Visual Token Compression Methods [54.4711434793961]
  We show that simple image downsampling consistently outperforms many advanced compression methods across multiple widely used benchmarks.<n>Motivated by these findings, we introduce VTC-Bench, an evaluation framework that incorporates a data filtering mechanism to denoise existing benchmarks.
  arXiv  Detail & Related papers  (2025-10-08T15:44:28Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search [61.4807238517108]
  Chain-of-Thought (CoT) reasoning enhances large language models (LLMs) by enabling step-by-step problem-solving.<n>CoT's extension to Long-CoT introduces substantial computational overhead due to increased token length.<n>We propose R1-Compress, a two-stage chunk-level compression framework that preserves both local information and coherence.
  arXiv  Detail & Related papers  (2025-05-22T16:06:59Z)
• ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference [61.412894960600205]
  Large Language Models (LLMs) require significant GPU memory when processing long texts.<n>ChunkKV reimagines KV cache compression by treating semantic chunks as basic compression units.<n>Result: ChunkKV outperforms state-of-the-art methods by up to 8.7% in precision.
  arXiv  Detail & Related papers  (2025-02-01T03:49:47Z)
• LeCo: Lightweight Compression via Learning Serial Correlations [9.108815508920882]
  Lightweight data compression is a key technique that allows column stores to exhibit superior performance for analytical queries. We propose LeCo (i.e., Learned Compression), a framework that uses machine learning to remove the serial redundancy in a value sequence automatically. We observe up to 5.2x speed up in a data analytical query in the Arrow columnar execution engine and a 16% increase in RocksDB's throughput.
  arXiv  Detail & Related papers  (2023-06-27T10:46:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.