FinCacheServe: Dependency-Consistent Answer Reuse for Cost-Efficient RAG Serving over Mutable Enterprise Documents

• URL: http://arxiv.org/abs/2607.26076v1
• Date: Tue, 14 Jul 2026 02:58:16 GMT
• Title: FinCacheServe: Dependency-Consistent Answer Reuse for Cost-Efficient RAG Serving over Mutable Enterprise Documents
• Abstract summary: FinCacheServe treats each generated answer as a serving object indexed by enterprise intent.<n>A vLLM implementation evaluates SEC-derived financial-document workloads with Qwen2.5 models.
• Score: 2.7688478017544274
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Retrieval-augmented generation services over mutable enterprise documents repeatedly execute semantically equivalent analysis requests. Answer reuse can remove GPU-bound generation work, yet response caches require dependency consistency when filings, evidence chunks, and tool outputs change. FinCacheServe treats each generated answer as a serving object indexed by enterprise intent and guarded by document versions, evidence fingerprints, tool fingerprints, model identity, and decoding configuration. A vLLM implementation evaluates SEC-derived financial-document workloads with Qwen2.5 models. On a 2,230-request hosted 7B trace, FinCacheServe skips 53.27% of LLM calls with zero observed dependency-stale outputs. Across three hosted 32B operator-suite seeds, it skips 53.31% of 544 requests, compared with 38.97% for versioned semantic caching and 22.43% for grounded-style reuse. Capacity, backend, and SLO replays show oracle-bounded cache management, 100k-entry transactional metadata behavior, and 44.30% lower estimated Wh per dependency-fresh 2s-SLO success than versioned semantic caching.

Related papers

• Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving [0.0]
  Prefix caching, in which a serving engine reuses the key and value tensors of a shared prompt prefix across requests, is enabled by default in the major open-source stacks.<n>We run an eighty-episode multi-turn agentic tool-use workload with caching enabled and disabled across two engines and four weight formats.<n>We find that repeated execution was bit-identical in every configuration, 0 of 800 episodes, which bounds other sources of nondeterminism at 0.5 percent.
  arXiv  Detail & Related papers  (2026-09-04T05:27:34Z)
• VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval [65.89247522243959]
  VisDocAgentBench is a closed-corpus benchmark comparing static and agentic retrieval under a ranked-output contract.<n>It contains 2,375 pages from 100 documents and 120 unique-target queries balanced across direct, one-bridge, and two-bridge evidence structures.<n>A strong late-interaction visual retriever reaches 97.50% Recall@1 on direct items but 2.50% on two-bridge items, exposing the limits of query--target matching when relevance depends on corpus context.
  arXiv  Detail & Related papers  (2026-08-18T15:23:06Z)
• KeyPooling: Measuring Where LLM API Relay Paths Collapse Prompt Cache Isolation [53.952294884822955]
  Large language model (LLM) API relays authenticate customers separately but often forward requests through shared provider credentials.<n>We present KeyPooling, a measurement method that traces customer identity through cache lookup and write.<n>We derive a defense contract: every customer must enter a provider-enforced domain.
  arXiv  Detail & Related papers  (2026-08-18T08:12:34Z)
• Learning Agent Execution for KV-Cache Management in Agentic Serving [25.021363822677515]
  We present CacheScout, an agent-aware KV-cache runtime layer for multi-agent LLM serving.<n>The key insight is that future KV-cache reuse is governed by agent execution semantics rather than cache recency alone.<n>We show that CacheScout improves KV-cache hit rate by 10-18 percentage points, reduces mean TTFT by 18-45%, lowers mean per-turn latency by 29-38%, and increases peak throughput by up to 57%.
  arXiv  Detail & Related papers  (2026-07-16T22:28:19Z)
• MEME: Multi-entity & Evolving Memory Evaluation [76.57263966646404]
  MEME defines six tasks spanning the full space defined by the multi-entity and evolving axes.<n> evaluating six memory systems spanning three memory paradigms on 100 controlled episodes.<n>We find that all systems collapse on dependency reasoning under the default configuration.
  arXiv  Detail & Related papers  (2026-05-12T17:55:10Z)
• StepCache: Step-Level Reuse with Lightweight Verification and Selective Patching for LLM Serving [0.0]
  We present StepCache, a backend-agnostic step-level reuse layer that segments outputs into ordered steps.<n>StepCache regenerates only failing regions via selective patching.<n>It reduces mean latency from 2.13 s to 0.67 s, median latency from 2.42 s to 0.01 s, and p95 latency from 3.38 s to 3.30 s.
  arXiv  Detail & Related papers  (2026-03-24T17:19:26Z)
• Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks [1.2292307778008844]
  We present a comprehensive evaluation of prompt caching across three major Large Language Model (LLM) providers.<n>Our results demonstrate that prompt caching reduces API costs by 45-80% and improves time to first token by 13-31% across providers.
  arXiv  Detail & Related papers  (2026-01-09T18:41:57Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
  We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models. We propose an importance-driven cache merging strategy to prune redundancy caches. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
  arXiv  Detail & Related papers  (2024-07-25T15:29:05Z)
• MeanCache: User-Centric Semantic Caching for LLM Web Services [8.350378532274405]
  Caching is a natural solution to reduce inference costs on repeated queries.<n>This paper introduces MeanCache, a user-centric semantic cache for LLM-based services.<n>MeanCache identifies semantically similar queries to determine cache hit or miss.
  arXiv  Detail & Related papers  (2024-03-05T06:23:50Z)
• DSI++: Updating Transformer Memory with New Documents [95.70264288158766]
  We introduce DSI++, a continual learning challenge for DSI to incrementally index new documents. We show that continual indexing of new documents leads to considerable forgetting of previously indexed documents. We introduce a generative memory to sample pseudo-queries for documents and supplement them during continual indexing to prevent forgetting for the retrieval task.
  arXiv  Detail & Related papers  (2022-12-19T18:59:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.