CacheProbe: Auditing Prompt Cache Isolation in Gateway APIs

• URL: http://arxiv.org/abs/2605.30613v1
• Date: Thu, 28 May 2026 22:06:54 GMT
• Title: CacheProbe: Auditing Prompt Cache Isolation in Gateway APIs
• Abstract summary: This paper investigates whether Open's API gateway architecture introduces prompt caching vulnerabilities that bypass provider-level prompt cache isolation guarantees.<n>Most inference providers implement per-account or per-organization prompt caching to prevent data leaks, but does routing through Open inadvertently create global cache sharing across all Open users?
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Over the past year, prompt caching in Large Language Models (LLMs) has become increasingly more popular across inference APIs. Prompt caching helps save precious compute resources and speeds up response times by reusing parts of the KV cache of a specific prompt for another request. However, many implementations of prompt caching are not secure against timing attacks or even basic metadata disclosure. Gu et al. (ICML 2025) develop a method to audit prompt caching in LLMs. This paper investigates whether OpenRouter's API gateway architecture introduces prompt caching vulnerabilities that bypass provider-level prompt cache isolation guarantees. Most LLM inference providers implement per-account or per-organization prompt caching to prevent data leaks, but does routing through OpenRouter with shared organizational credentials inadvertently create global cache sharing across all OpenRouter users?

Related papers

• KeyPooling: Measuring Where LLM API Relay Paths Collapse Prompt Cache Isolation [53.952294884822955]
  Large language model (LLM) API relays authenticate customers separately but often forward requests through shared provider credentials.<n>We present KeyPooling, a measurement method that traces customer identity through cache lookup and write.<n>We derive a defense contract: every customer must enter a provider-enforced domain.
  arXiv  Detail & Related papers  (2026-08-18T08:12:34Z)
• LaCache: Robust Semantic Caching for LLM Serving [10.741523413040559]
  We present LaCache, a novel semantic caching scheme that addresses cache-collision attacks.<n>LaCache checks the cache hit of its first k decoded tokens rather than checking only the cache hit of a query.<n>We prove that it is impossible to craft adversarial queries that simultaneously elicit malicious responses and collide with benign queries.
  arXiv  Detail & Related papers  (2026-08-03T05:31:24Z)
• Back from the Future: Key-Value Cache Management by Counter-Causal Surprise [67.1056509495879]
  Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.<n>We propose a simple yet effective KV eviction scheme motivated by the insight that past tokens which can be well-predicted from more recent tokens are redundant.<n>We evaluate our strategy on various open-source LLMs and benchmark datasets showing competitive or improved performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-30T02:42:51Z)
• Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks [1.2292307778008844]
  We present a comprehensive evaluation of prompt caching across three major Large Language Model (LLM) providers.<n>Our results demonstrate that prompt caching reduces API costs by 45-80% and improves time to first token by 13-31% across providers.
  arXiv  Detail & Related papers  (2026-01-09T18:41:57Z)
• Breaking Diffusion with Cache: Exploiting Approximate Caches in Diffusion Models [1.399348653165494]
  We introduce a prompt stealing attack using the cache, where an attacker can recover existing cached prompts based on cache hit prompts.<n>We introduce a poisoning attack that embeds the attacker's logos into the previously stolen prompt, to render them in future user prompts that hit the cache.<n>These attacks are all performed remotely through the serving system, which indicates severe security vulnerabilities in approximate caching.
  arXiv  Detail & Related papers  (2025-08-28T04:46:44Z)
• Auditing Prompt Caching in Language Model APIs [77.02079451561718]
  We investigate the privacy leakage caused by prompt caching in large language models (LLMs)<n>We detect global cache sharing across users in seven API providers, including OpenAI.<n>We find evidence that OpenAI's embedding model is a decoder-only Transformer, which was previously not publicly known.
  arXiv  Detail & Related papers  (2025-02-11T18:58:04Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
  We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models. We propose an importance-driven cache merging strategy to prune redundancy caches. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
  arXiv  Detail & Related papers  (2024-07-25T15:29:05Z)
• Prompt Cache: Modular Attention Reuse for Low-Latency Inference [12.610067639587461]
  We present Prompt Cache, an approach for accelerating inference for large language models (LLM) by reusing attention states across different prompts. Prompt Cache employs a schema to explicitly define such reusable text segments, called prompt modules. We show that Prompt Cache significantly reduce latency in time-to-first-token, especially for longer prompts.
  arXiv  Detail & Related papers  (2023-11-07T18:17:05Z)
• Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching [72.50506500576746]
  We propose a novel caching paradigm, that we named approximate-key caching. While approximate cache hits alleviate DL inference workload and increase the system throughput, they however introduce an approximation error. We analytically model our caching system performance for classic LRU and ideal caches, we perform a trace-driven evaluation of the expected performance, and we compare the benefits of our proposed approach with the state-of-the-art similarity caching.
  arXiv  Detail & Related papers  (2021-12-13T13:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.