RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

• URL: http://arxiv.org/abs/2608.01247v1
• Date: Sun, 02 Aug 2026 13:58:42 GMT
• Title: RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
• Abstract summary: KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets.<n>We introduce RestoreKV, which complements this selection-based formulation with learned restoration under the same total KV budget.<n>Our key insight is that, although the information lost through eviction is context-specific, the mechanism for generating its compact complement can be shared across contexts.
• Score: 10.668136973950453
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets. Existing methods primarily improve which original KV pairs are retained. We introduce RestoreKV, which complements this selection-based formulation with learned restoration under the same total KV budget. Our key insight is that, although the information lost through eviction is context-specific, the mechanism for generating its compact complement can be shared across contexts. After context prefill, a few restore tokens attend to the full KV cache in a single LoRA-adapted pass, generating a compact, context-conditioned restore cache. The base importance scorer and eviction rule remain unchanged, and the adapters are disabled for all subsequent queries and decoding. RestoreKV is trained through parameter-efficient self-distillation from the frozen full-cache model, optimizing only $0.4\%$ of the parameters and requiring no task-specific tuning. Across four backbones and four long-context benchmarks, RestoreKV substantially reduces compression-induced degradation. On Qwen3-4B, it improves 59 of 60 paired, budget-matched settings across five base eviction methods; at a $5\%$ budget, it raises KVzip from $38.2$ to $73.2$ on RULER-4K. Applied to KVzip+, RestoreKV reaches $86.4$ RULER accuracy at $16\times$ compression on the KVPress Benchmark, while adding less than $0.5\%$ one-time cache-construction overhead in a 32K-context evaluation. Our project page is available at https://paper.pnu-cvsp.com/RestoreKV/

Related papers

• S$^4$R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching [38.29032214906932]
  We propose S$4$R, which builds low-rank subspaces from selectively sampled tokens and computes attention over a sparsely reconstructed KV representation.<n>Experiments on LongBench and RULER with Llama and Qwen model families show that S$4$R achieves up to 5$times$ KV compression with near full-cache accuracy.
  arXiv  Detail & Related papers  (2026-08-01T08:41:29Z)
• C$^2$KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference [31.428047145437773]
  Long-context inference is central to modern large language model (LLM) applications such as retrieval-augmented generation and multi-document reasoning.<n>To mitigate the growing inference cost, recent work has explored key-value ( KV) cache reuse to reduce redundant prefill computation.<n>Existing reuse methods primarily focus on savings and overlook a critical bottleneck in long-context serving: the cost of storing and accessing KV caches.
  arXiv  Detail & Related papers  (2026-07-20T09:09:23Z)
• GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs [97.36238579001544]
  Large language models (LLMs) with extended context lengths rely on the key-value ( KV) cache to support attention over prior tokens.<n>Maintaining the KV cache incurs substantial memory overhead, motivating KV-cache compression methods.<n>We propose GRKV, a training-free KV-cache merging method that directly minimizes the discrepancy between compressed-cache and full-cache attention outputs.
  arXiv  Detail & Related papers  (2026-05-29T10:16:30Z)
• Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction [65.710271475739]
  We introduce a global retention-based KV eviction method that learns each token's future utility under a unified memory budget.<n>We show that our method substantially reduces KV memory while matching or surpassing full-cache inference.<n>These results suggest that learned, globally calibrated KV eviction is not only a compression technique, but also a mechanism for improving long-context reasoning.
  arXiv  Detail & Related papers  (2026-05-10T16:47:50Z)
• The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference [3.378773775514883]
  Key-value (KV) cache is widely treated as essential state in transformer inference.<n>We prove that keys and values at every layer are deterministic projections of the residual stream.<n>We build on this result with KV-Direct, a bounded-memory inference scheme.
  arXiv  Detail & Related papers  (2026-03-20T05:59:50Z)
• KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction [20.53279247581787]
  We propose KVReviver, a reversible KV cache compression method based on the sketch algorithm.<n>In 2k-length contexts, it requires only 10% of KV Cache budget while maintaining identical end-to-end inference accuracy.<n>For 32k-length contexts, it achieves equivalent or comparable accuracy 2% accuracy loss.
  arXiv  Detail & Related papers  (2025-12-01T03:59:20Z)
• ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibration [69.57122277845293]
  We propose ReCalKV, a post-training low-rank KV cache compression approach with tailored strategies for Keys and Values.<n>For Keys, we propose Similarity aware Recontext (HSR), which clusters structurally similar heads into groups, enabling more accurate low-rank approximation.<n>For Values, we propose Offline Head-wise Value (OVC), which efficiently calibrates the value projection matrix using calibration data without training.
  arXiv  Detail & Related papers  (2025-05-30T08:49:27Z)
• R-KV: Redundancy-aware KV Cache Compression for Reasoning Models [77.84539432982307]
  We propose Redundancy-aware KV Cache Compression for Reasoning models (R-KV)<n>R-KV preserves nearly 100% of the full KV cache performance using only 10% of the KV cache.<n>Remarkably, R-KV even achieves 105% of full KV cache performance with 16% of the KV cache.
  arXiv  Detail & Related papers  (2025-05-30T02:03:24Z)
• KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction [37.97434606840326]
  Transformer-based large language models (LLMs) cache context as key-value ( KV) pairs during inference.<n>As context length grows, KV cache sizes expand, leading to substantial memory overhead and increased attention latency.<n>This paper introduces KVzip, a query-agnostic KV cache eviction method enabling effective reuse of compressed KV caches across diverse queries.
  arXiv  Detail & Related papers  (2025-05-29T13:05:47Z)
• FlowKV: Enhancing Multi-Turn Conversational Coherence in LLMs via Isolated Key-Value Cache Management [48.904743679691414]
  FlowKV is a novel multi-turn isolation mechanism for KV Cache management.<n>It preserves the accumulated compressed KV cache from past turns.<n>It prevents the re-compression of older context and thereby mitigating catastrophic forgetting.
  arXiv  Detail & Related papers  (2025-05-21T10:20:46Z)
• DBudgetKV: Dynamic Budget in KV Cache Compression for Ensuring Optimal Performance [125.81664663201282]
  We introduce a new KV cache compression method dubbed DBudgetKV.<n>It features an attention-based metric to signal when the remaining KV cache is unlikely to match the full-cache performance.<n>Our method achieves lossless KV pruning effectively and robustly, exceeding 25% compression ratio on average.
  arXiv  Detail & Related papers  (2025-02-24T06:33:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.