Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?

• URL: http://arxiv.org/abs/2605.27494v1
• Date: Tue, 26 May 2026 16:50:02 GMT
• Title: Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?
• Abstract summary: We propose GroundedCache, an evidence-validated cache router that admits a cached answer only when 4 cheap gates simultaneously hold.<n>We build a six-regime workload that stress-tests cache safety rather than only hit rate, and introduce an operator-facing metric, the unsafe-served rate (USR)<n>Across 2 datasets and 12,000 real-LLM generations(Qwen2.5-7B-Instruct on vLLM with Automatic Prefix Caching), GroundedCache drives USR to 0.0% on every HotpotQA regime and to 1.5% on mtRAG document drift.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern retrieval-augmented generation(RAG) deployments increasingly rely on caching to reduce token cost and time-to-first-token(TTFT). Prefix-level KV reuse is now standard in serving stacks such as vLLM, and chunk-level and position-independent reuse have been pushed further by recent systems(RAGCache, TurboRAG, CacheBlend, EPIC, ContextPilot, PCR, LMCache). Output-level semantic answer caches, by contrast, remain fragile: similar prompts can map to different correct answers, retrieved evidence drifts as the corpus is updated, and adversarial collision attacks have been shown to hijack cached responses. We argue that the right framing for cached answer reuse is not how to reuse faster but when reuse is safe. We propose GroundedCache, an evidence-validated cache router that admits a cached answer only when 4 cheap gates simultaneously hold: query similarity, retrieved-evidence overlap, source-version validity, and lexical (or judge-based) support of the cached answer by the freshly retrieved evidence. We build a six-regime workload that stress-tests cache safety rather than only hit rate, and introduce an operator-facing metric, the unsafe-served rate (USR), fraction of all queries that received a wrong cached answer. Across 2 datasets and 12,000 real-LLM generations(Qwen2.5-7B-Instruct on vLLM with Automatic Prefix Caching), GroundedCache drives USR to 0.0% on every HotpotQA regime(vs. 15-35% under naive caching) and to 1.5% on mtRAG document drift(vs. 51.5%), a 34x reduction on the design-point adversarial regime and 3-10x reductions across the other mtRAG regimes, while end-to-end p50 latency stays within 1.04-1.07x of a no-cache RAG baseline. A per-gate ablation isolates the lexical support gate as the load-bearing safety mechanism on both datasets, with the remaining gates providing defense-in-depth at near-zero cost. We release the implementation, workload, and evaluation harness.

Related papers

• KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints [14.86076891340077]
  KVShare-cache reuse across prompt contexts and model checkpoints on retrieved chunks and agent reports.<n>We find that correcting positions, which needs no recomputation, is enough until a question needs several sources at once.<n> KVArena-cache scores every method by the fraction of the gap it recovers between no cache and full recomputation.
  arXiv  Detail & Related papers  (2026-09-09T14:53:45Z)
• Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference [0.0]
  Key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference.<n>Three published attacks exploit it, reaching up to 100% attack success rate against unprotected vLLM and SG.<n>We present KVGov, a governance layer addressing all three attack families' prefix-cache paths.
  arXiv  Detail & Related papers  (2026-08-10T07:47:57Z)
• LaCache: Robust Semantic Caching for LLM Serving [10.741523413040559]
  We present LaCache, a novel semantic caching scheme that addresses cache-collision attacks.<n>LaCache checks the cache hit of its first k decoded tokens rather than checking only the cache hit of a query.<n>We prove that it is impossible to craft adversarial queries that simultaneously elicit malicious responses and collide with benign queries.
  arXiv  Detail & Related papers  (2026-08-03T05:31:24Z)
• Back from the Future: Key-Value Cache Management by Counter-Causal Surprise [67.1056509495879]
  Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.<n>We propose a simple yet effective KV eviction scheme motivated by the insight that past tokens which can be well-predicted from more recent tokens are redundant.<n>We evaluate our strategy on various open-source LLMs and benchmark datasets showing competitive or improved performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-30T02:42:51Z)
• Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning [51.563653495547335]
  multimodal reasoning improves visual grounding by revisiting visual evidence during multi-step generation.<n>We propose Position Rebinding Cache Reuse (PRCR), a cache-level framework for replay-free visual revisiting.<n>PRCR achieves replay-level or better performance, improving average accuracy by 5 percent and reducing visual-revisiting by up to tens of thousands of times.
  arXiv  Detail & Related papers  (2026-06-25T05:47:52Z)
• CacheWeaver: Cache-Aware Evidence Ordering for Efficient Grounded RAG Inference [4.3361375690943795]
  Prefix caching in serving engines such as vLLM reduces this cost only when requests share the same token prefix.<n>We present CacheWeaver, a lightweight prompt-layer method for cache-aware evidence ordering.
  arXiv  Detail & Related papers  (2026-06-18T00:38:46Z)
• Sparse Prefix Caching for Hybrid and Recurrent LLM Serving [0.7284556903703034]
  We formalize sparse prefix caching as checkpoint placement under a distribution over overlap depths.<n>For use cases where requests share a non-trivial prefix, we show that our method consistently improves the frontier traced by standard asymmetry on real-world data.<n>It preserves exact outputs, does not change the recurrent computation itself or require new recurrent update kernels.
  arXiv  Detail & Related papers  (2026-04-17T09:24:58Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• dKV-Cache: The Cache for Diffusion Language Models [53.85291644298835]
  Diffusion Language Models (DLMs) have been seen as a promising competitor for autoregressive language models.<n>We propose a KV-cache-like mechanism, delayed KV-Cache, for the denoising process of DLMs.<n>Our approach is motivated by the observation that different tokens have distinct representation dynamics throughout the diffusion process.
  arXiv  Detail & Related papers  (2025-05-21T17:32:10Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
  We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models. We propose an importance-driven cache merging strategy to prune redundancy caches. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
  arXiv  Detail & Related papers  (2024-07-25T15:29:05Z)
• Training-Free Exponential Context Extension via Cascading KV Cache [49.608367376911694]
  We introduce a novel mechanism that leverages cascading sub-cache buffers to selectively retain the most relevant tokens.<n>Our method reduces prefill stage latency by a factor of 6.8 when compared to flash attention on 1M tokens.
  arXiv  Detail & Related papers  (2024-06-24T03:59:17Z)
• BackCache: Mitigating Contention-Based Cache Timing Attacks by Hiding Cache Line Evictions [7.46215723037597]
  L1 data cache attacks pose a significant privacy and confidentiality threat. BackCache always achieves cache hits instead of cache misses to mitigate contention-based cache timing attacks on the L1 data cache. BackCache places the evicted cache lines from the L1 data cache into a fully-associative backup cache to hide the evictions.
  arXiv  Detail & Related papers  (2023-04-20T12:47:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.