Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference

• URL: http://arxiv.org/abs/2608.09225v1
• Date: Mon, 10 Aug 2026 07:47:57 GMT
• Title: Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference
• Abstract summary: Key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference.<n>Three published attacks exploit it, reaching up to 100% attack success rate against unprotected vLLM and SG.<n>We present KVGov, a governance layer addressing all three attack families' prefix-cache paths.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference, enabling prefix reuse across requests. In multi-tenant deployments this cache is shared across tenants, creating a timing side channel: an adversarial tenant can reconstruct another tenant's private prompt by probing cache-hit latency. Three published attacks exploit it -- PROMPTPEEK, EarlyBird and InputSnatch -- reaching up to 100% attack success rate against unprotected vLLM and SGLang, with rates varying by cache architecture and prompt structure. We present KVGov, a governance layer addressing all three attack families' prefix-cache paths under one mechanism. A per-principal salt sigma_p = HMAC_K(secret, principal_id) seeds the block-hash chain, making cache keys cryptographically disjoint across principals. An ablation (N=1000 trials, seed 2026, deterministic judges) isolates this salt as the necessary and sufficient component. KVGov adds ORIGAMI, a Stackelberg water-filling audit scheduler that reduces adversary expected utility by 12.6% at realistic tenant heterogeneity (Gini 0.63), and an evolutionary stability analysis giving a 31.6% adversary-prevalence tipping point below which global caching remains stable. On real hardware (Qwen2.5-7B-Instruct, vLLM 0.26.0, NVIDIA A100) we measure a gate-verified cold/cached TTFT ratio of 0.22, confirming the channel is exploitable at production scale; the defense itself is evaluated in simulation calibrated to those measurements. We replicate the channel on an independent stack (llama.cpp on Apple Metal, ratio 0.093). Finally, isolation and cache efficiency need not conflict: identifying information resides only where prompts diverge, so injecting the salt at that boundary rather than the chain root retains an estimated 93% of the prefix-cache benefit with no cross-principal signal.

Related papers

• KeyPooling: Measuring Where LLM API Relay Paths Collapse Prompt Cache Isolation [53.952294884822955]
  Large language model (LLM) API relays authenticate customers separately but often forward requests through shared provider credentials.<n>We present KeyPooling, a measurement method that traces customer identity through cache lookup and write.<n>We derive a defense contract: every customer must enter a provider-enforced domain.
  arXiv  Detail & Related papers  (2026-08-18T08:12:34Z)
• Back from the Future: Key-Value Cache Management by Counter-Causal Surprise [67.1056509495879]
  Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.<n>We propose a simple yet effective KV eviction scheme motivated by the insight that past tokens which can be well-predicted from more recent tokens are redundant.<n>We evaluate our strategy on various open-source LLMs and benchmark datasets showing competitive or improved performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-30T02:42:51Z)
• Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG [17.045113249152234]
  We introduce the first end-to-end side-channel attack targeting non- KV cache fusion.<n>SpliceLeak executes a systematic two-phase privacy breach.<n>SpliceDefense effectively flattens the side-channel signal (Delta TTFT 0) with negligible throughput overhead.
  arXiv  Detail & Related papers  (2026-06-20T02:28:28Z)
• RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference [0.0]
  We introduce RKSC (Reasoning-Aware KV Cache Sharing), a training-free inference framework.<n>It eliminates two structural redundancies in multi-branch LLM reasoning pipelines.<n>RKSC achieves a mean speedup of 3.008x over the No-KV baseline.
  arXiv  Detail & Related papers  (2026-06-07T21:36:20Z)
• Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer? [0.0]
  We propose GroundedCache, an evidence-validated cache router that admits a cached answer only when 4 cheap gates simultaneously hold.<n>We build a six-regime workload that stress-tests cache safety rather than only hit rate, and introduce an operator-facing metric, the unsafe-served rate (USR)<n>Across 2 datasets and 12,000 real-LLM generations(Qwen2.5-7B-Instruct on vLLM with Automatic Prefix Caching), GroundedCache drives USR to 0.0% on every HotpotQA regime and to 1.5% on mtRAG document drift.
  arXiv  Detail & Related papers  (2026-05-26T16:50:02Z)
• Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction [65.710271475739]
  We introduce a global retention-based KV eviction method that learns each token's future utility under a unified memory budget.<n>We show that our method substantially reduces KV memory while matching or surpassing full-cache inference.<n>These results suggest that learned, globally calibrated KV eviction is not only a compression technique, but also a mechanism for improving long-context reasoning.
  arXiv  Detail & Related papers  (2026-05-10T16:47:50Z)
• Latent-Condensed Transformer for Efficient Long Context Modeling [60.72493959155964]
  We propose Latent-Condensed Attention (LCA) for large language models.<n>LCA condenses context within MLA's latent space, where representation is disentangled into semantic latent vectors and positional keys.<n>LCA achieves up to 2.5$times$ prefilling speedup and 90% KV cache reduction at 128K context.
  arXiv  Detail & Related papers  (2026-04-14T08:40:31Z)
• CacheClip: Accelerating RAG with Effective KV Cache Reuse [8.016679032026824]
  CacheClip is a novel framework that achieves both fast TTFT and high generation quality.<n>It integrates three techniques: (1) auxiliary-model-guided token selection for selective KV cache recomputation, where the auxiliary model is finetuned to improve selection accuracy, (2) shared prefixes to eliminate redundant attention sinks, and (3) grouping strategy to maintain local coherence.
  arXiv  Detail & Related papers  (2025-10-11T09:28:26Z)
• Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference [6.864810630905683]
  Existing defenses, such as per-user isolation, eliminate leakage but degrade performance by up to 38.9% in time-to-first-token (TTFT)<n>We introduce SafeKV, a privacy-aware KV-cache management framework that selectively shares non-sensitive entries while confining sensitive content to private caches.<n>Our evaluation shows that SafeKV mitigates 94% - 97% of timing-based side-channel attacks.
  arXiv  Detail & Related papers  (2025-08-11T19:55:44Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
  We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models. We propose an importance-driven cache merging strategy to prune redundancy caches. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
  arXiv  Detail & Related papers  (2024-07-25T15:29:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.