AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents

• URL: http://arxiv.org/abs/2604.24039v1
• Date: Mon, 27 Apr 2026 04:51:15 GMT
• Title: AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents
• Abstract summary: Embodied AI agents increasingly rely on large language models (LLMs) for planning, yet per-step LLM calls impose severe latency and cost.<n>We introduce AgenticCache, a framework that reuses cached plans to avoid per-step LLM calls.<n>AgenticCache improves task success rate by 22% on average across 12 configurations, reduces simulation latency by 65%, and lowers token usage by 50%.
• Score: 3.275342475988451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied AI agents increasingly rely on large language models (LLMs) for planning, yet per-step LLM calls impose severe latency and cost. In this paper, we show that embodied tasks exhibit strong plan locality, where the next plan is largely predictable from the current one. Building on this, we introduce AgenticCache, a planning framework that reuses cached plans to avoid per-step LLM calls. In AgenticCache, each agent queries a runtime cache of frequent plan transitions, while a background Cache Updater asynchronously calls the LLM to validate and refine cached entries. Across four multi-agent embodied benchmarks, AgenticCache improves task success rate by 22% on average across 12 configurations (4 benchmarks x 3 models), reduces simulation latency by 65%, and lowers token usage by 50%. Cache-based plan reuse thus offers a practical path to low-latency, low-cost embodied agents. Code is available at https://github.com/hojoonleokim/MLSys26_AgenticCache.

Related papers

• Learning Agent Execution for KV-Cache Management in Agentic Serving [25.021363822677515]
  We present CacheScout, an agent-aware KV-cache runtime layer for multi-agent LLM serving.<n>The key insight is that future KV-cache reuse is governed by agent execution semantics rather than cache recency alone.<n>We show that CacheScout improves KV-cache hit rate by 10-18 percentage points, reduces mean TTFT by 18-45%, lowers mean per-turn latency by 29-38%, and increases peak throughput by up to 57%.
  arXiv  Detail & Related papers  (2026-07-16T22:28:19Z)
• MiniCache: Reusable Program Caching with Small Model Interfaces for Efficient LLM Inference [8.76886279227683]
  MiniCache is a reusable program caching framework that transforms Program-of-Thought (PoT) programs into parameterized cache objects.<n>MiniCache reuses the same small model for semantic variable extraction on cache-hit requests and speculative drafting during target-LLM generation.<n>Experiments on shopping-style request datasets, WebShop, Formula, and CodeTAT-QA demonstrate that MiniCache improves the trade-off between inference latency, cache reuse, and accuracy.
  arXiv  Detail & Related papers  (2026-07-03T06:53:48Z)
• Towards Distributed Inference of LLMs on a P2P Network [1.1458853556386797]
  Prefix caching can reduce LLM inference latency by reusing KV caches across requests with shared prompts.<n>We propose a decentralized, prefix-cache-aware routing scheme for peer-to-peer LLM serving.
  arXiv  Detail & Related papers  (2026-05-07T15:40:51Z)
• SPA-Cache: Singular Proxies for Adaptive Caching in Diffusion Language Models [56.45983529954998]
  We present SPA-Cache that jointly optimize update identification and budget allocation in DLM cache.<n>First, we derive a low-dimensional singular proxy that enables the identification of update-critical tokens in a low-dimensional subspace.<n>Second, we introduce an adaptive strategy that allocates fewer updates to stable layers without degrading generation quality.
  arXiv  Detail & Related papers  (2026-01-30T05:22:44Z)
• TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving [12.80179556886128]
  We propose a unified segment-level prefix cache pool, TokenLake.<n>It uses a cache interface to expose requests' query tensors, prefixes, and cache-aware operations.<n> TokenLake can improve throughput by up to 2.6$times$ and 2.0$times$ and boost hit rate by 2.0$times$ and 2.1$times$.
  arXiv  Detail & Related papers  (2025-08-24T05:45:16Z)
• LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models [52.56008278458534]
  LaCache is a training-free method for efficient and accurate generative inference of Large Language Models.<n>LaCache enables LLMs to address both of the critical challenges in long-range modeling: robust long-range capabilities and continuous generation without running out-of-memory.
  arXiv  Detail & Related papers  (2025-07-14T19:09:57Z)
• dKV-Cache: The Cache for Diffusion Language Models [53.85291644298835]
  Diffusion Language Models (DLMs) have been seen as a promising competitor for autoregressive language models.<n>We propose a KV-cache-like mechanism, delayed KV-Cache, for the denoising process of DLMs.<n>Our approach is motivated by the observation that different tokens have distinct representation dynamics throughout the diffusion process.
  arXiv  Detail & Related papers  (2025-05-21T17:32:10Z)
• Enhancing Lifelong Multi-Agent Path Finding with Cache Mechanism [25.575748142768298]
  Lifelong MAPF (L-MAPF) offers a more realistic approximation of real-world warehouse scenarios.<n>We introduce a novel mechanism called Lifelong MAPF with Cache Mechanism (L-MAPF-CM), which integrates high-level cache storage with low-level path planning.<n>L-MAPF-CM has demonstrated performance improvements particularly with high cache hit rates and smooth traffic conditions.
  arXiv  Detail & Related papers  (2025-01-06T06:44:13Z)
• InstCache: A Predictive Cache for LLM Serving [6.076957323090607]
  Caching techniques offer opportunities to optimize the performance of Large Language Models inference engines.<n>High variability in the content and length of instructions make it rare for identical instructions to recur within a short time window.<n>We propose InstCache, a predictive caching mechanism for LLM serving systems.
  arXiv  Detail & Related papers  (2024-11-21T03:52:41Z)
• Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
  We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models. We propose an importance-driven cache merging strategy to prune redundancy caches. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
  arXiv  Detail & Related papers  (2024-07-25T15:29:05Z)
• Training-Free Exponential Context Extension via Cascading KV Cache [49.608367376911694]
  We introduce a novel mechanism that leverages cascading sub-cache buffers to selectively retain the most relevant tokens.<n>Our method reduces prefill stage latency by a factor of 6.8 when compared to flash attention on 1M tokens.
  arXiv  Detail & Related papers  (2024-06-24T03:59:17Z)
• MeanCache: User-Centric Semantic Caching for LLM Web Services [8.350378532274405]
  Caching is a natural solution to reduce inference costs on repeated queries.<n>This paper introduces MeanCache, a user-centric semantic cache for LLM-based services.<n>MeanCache identifies semantically similar queries to determine cache hit or miss.
  arXiv  Detail & Related papers  (2024-03-05T06:23:50Z)
• NumS: Scalable Array Programming for the Cloud [82.827921577004]
  We present NumS, an array programming library which optimize NumPy-like expressions on task-based distributed systems. This is achieved through a novel scheduler called Load Simulated Hierarchical Scheduling (LSHS) We show that LSHS enhances performance on Ray by decreasing network load by a factor of 2x, requiring 4x less memory, and reducing execution time by 10x on the logistic regression problem.
  arXiv  Detail & Related papers  (2022-06-28T20:13:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.