KeyPooling: Measuring Where LLM API Relay Paths Collapse Prompt Cache Isolation

• URL: http://arxiv.org/abs/2608.17485v2
• Date: Sun, 23 Aug 2026 04:39:13 GMT
• Title: KeyPooling: Measuring Where LLM API Relay Paths Collapse Prompt Cache Isolation
• Abstract summary: Large language model (LLM) API relays authenticate customers separately but often forward requests through shared provider credentials.<n>We present KeyPooling, a measurement method that traces customer identity through cache lookup and write.<n>We derive a defense contract: every customer must enter a provider-enforced domain.
• Score: 53.952294884822955
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) API relays authenticate customers separately but often forward requests through shared provider credentials. Providers scope prompt caches to upstream principals and namespaces, so relay customers mapped to one cache identity can observe each other's cache state. Prior work showed cache sharing at selected endpoints but did not identify which credential, pool, adapter, or nested hop controls the finalidentity. We present KeyPooling, a measurement method that traces customer identity through cache lookup and write, verifies runtime transformations, and tests one predicted identity component at a time. Across five open-source gateways connected to OpenAI and Anthropic, none bound customers to upstream credentials by default; under a shared credential, all five exposed cross-customer cache reads for both providers. Principal and namespace splits, pool associations, and adapter and nested-relay contrasts localized the controlling transformations. In an outcome-independent weekly OpenRouter frame, tests covered 80.5% of eligible token volume and found cross-account reads for 12 of 28 labels carrying 33.7% of volume. On one production route, a controlled procedure recovered eight consecutive target positions without target access. Broader tests identify cache granularity, routing, rate limits, attribution, and budget as conditions for token-by-token recovery, not security controls. We derive a defense contract: every customer must enter a provider-enforced domain, or a namespace derived from authenticated identity must survive every final cache lookup and write. Placing this split after reusable public prefixes preserved most modeled reuse at a 1.7-2.5% cost increase.

Related papers

• ANCHOR: A Vision for Secure Persistent Key-Value Stores in Disaggregated Data Centers [5.04954154165985]
  ANCHOR is a vision for end-to-end integrity and freshness in disaggregated PKVS.<n>This paper presents ANCHOR, a vision for end-to-end integrity and freshness in disaggregated PKVS.
  arXiv  Detail & Related papers  (2026-08-28T01:37:51Z)
• Uncovering and Understanding Hidden Dependencies in the LLM API Reseller Ecosystem via Prefix-Cache Side Channels [14.313473801403404]
  LLM API resellers have become an important access layer to modern LLM services.<n>Existing studies audit individual resellers, but provide little visibility into hidden dependencies across resellers.<n>We present CacheTracer, the first API-only measurement of such hidden dependencies.
  arXiv  Detail & Related papers  (2026-08-21T04:35:21Z)
• Governing the KV Cache: Preventing Timing Side-Channel Leakage in Multi-Tenant LLM Inference [0.0]
  Key-value (KV) cache is the primary throughput optimization in modern large language model (LLM) inference.<n>Three published attacks exploit it, reaching up to 100% attack success rate against unprotected vLLM and SG.<n>We present KVGov, a governance layer addressing all three attack families' prefix-cache paths.
  arXiv  Detail & Related papers  (2026-08-10T07:47:57Z)
• The Trust-Free Aggregation Layer of the Unicity Infrastructure [1.1766230941766413]
  Unicity is a novel blockchain infrastructure for enabling users to execute off-chain peer-to-peer token transactions.<n>A key component of the infrastructure is the Aggregation Layer responsible for storing information about the spent states of tokens.<n>Our implementation uses no trusted setup, achieves throughput of 10,000 insertions per second and millisecond range verification time on a single consumer-class CPU.
  arXiv  Detail & Related papers  (2026-08-05T18:18:16Z)
• LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference [63.932718076408584]
  LAST is a training-free framework for query-dependent visual token pruning in edge-cloud collaborative MLLM inference.<n>It preserves 95.4% of the full-token accuracy while retaining only 12.5% of the visual tokens, with low edge-side selection overhead and reduced cloud-side computation.
  arXiv  Detail & Related papers  (2026-07-30T09:59:25Z)
• Back from the Future: Key-Value Cache Management by Counter-Causal Surprise [67.1056509495879]
  Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.<n>We propose a simple yet effective KV eviction scheme motivated by the insight that past tokens which can be well-predicted from more recent tokens are redundant.<n>We evaluate our strategy on various open-source LLMs and benchmark datasets showing competitive or improved performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-30T02:42:51Z)
• GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents [61.50171793000712]
  GateMem is a benchmark for multi-principal shared-memory agents.<n>It spans medical, office, education, and household domains.<n>No method simultaneously achieves strong utility, robust access control, and reliable forgetting.
  arXiv  Detail & Related papers  (2026-06-17T09:06:15Z)
• CacheProbe: Auditing Prompt Cache Isolation in Gateway APIs [0.0]
  This paper investigates whether Open's API gateway architecture introduces prompt caching vulnerabilities that bypass provider-level prompt cache isolation guarantees.<n>Most inference providers implement per-account or per-organization prompt caching to prevent data leaks, but does routing through Open inadvertently create global cache sharing across all Open users?
  arXiv  Detail & Related papers  (2026-05-28T22:06:54Z)
• StepCache: Step-Level Reuse with Lightweight Verification and Selective Patching for LLM Serving [0.0]
  We present StepCache, a backend-agnostic step-level reuse layer that segments outputs into ordered steps.<n>StepCache regenerates only failing regions via selective patching.<n>It reduces mean latency from 2.13 s to 0.67 s, median latency from 2.42 s to 0.01 s, and p95 latency from 3.38 s to 3.30 s.
  arXiv  Detail & Related papers  (2026-03-24T17:19:26Z)
• VA-DAR: A PQC-Ready, Vendor-Agnostic Deterministic Artifact Resolution for Serverless, Enumeration-Resistant Wallet Recovery [1.7429038786735553]
  VA-DAR is a keyed-discovery protocol for ACE-GF-based wallets.<n>It supports cross-device recovery using only a user-provided identifier and a single recovery passphrase.
  arXiv  Detail & Related papers  (2026-03-03T07:33:10Z)
• Auditing Prompt Caching in Language Model APIs [77.02079451561718]
  We investigate the privacy leakage caused by prompt caching in large language models (LLMs)<n>We detect global cache sharing across users in seven API providers, including OpenAI.<n>We find evidence that OpenAI's embedding model is a decoder-only Transformer, which was previously not publicly known.
  arXiv  Detail & Related papers  (2025-02-11T18:58:04Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.