Workload-Aware Caching for Multi-Agent Systems

• URL: http://arxiv.org/abs/2607.20495v1
• Date: Sun, 14 Jun 2026 02:13:27 GMT
• Title: Workload-Aware Caching for Multi-Agent Systems
• Abstract summary: Cache eviction policies treat all cached entries uniformly based on access history, ignoring structural and workload signals.<n>We present a workload-aware eviction policy that combines three signals, namely recomputation cost, DAG dependency count, and agent invocation frequency, into a unified scoring function.<n>Our policy reduces latency by up to 64.7% relative to the uncached baseline and achieves on average a 31.1% latency reduction over the next best finite-capacity baseline.
• Score: 3.2309496907010384
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent systems decompose complex tasks into directed acyclic graphs (DAGs) of specialized agent executions, creating natural opportunities for caching intermediate results across queries. However, existing cache eviction policies treat all cached entries uniformly based on access history, ignoring structural and workload signals uniquely available in agentic execution environments. We present a workload-aware eviction policy that combines three signals, namely recomputation cost, DAG dependency count, and agent invocation frequency, into a unified scoring function that retains the most valuable entries under memory constraints. Evaluated across three multi-agent benchmarks spanning diverse reuse regimes, our policy reduces latency by up to 64.7% relative to the uncached baseline and achieves on average a 31.1% latency reduction over the next best finite-capacity baseline, while approaching the performance of an unbounded cache and maintaining accuracy on par with or exceeding all competing finite-capacity methods. We further show that workload-aware content caching is complementary to other agentic system optimization methods, including plan-level caching and parallel agent execution, with each technique targeting a distinct efficiency bottleneck in multi-agent pipelines.

Related papers

• Evaluating Temporal Semantic Caching and Workflow Optimization in Agentic Plan-Execute Pipelines [3.4989760172592264]
  Industrial asset operations are latency-sensitive because a single user query may require coordination over sensor data, work orders, failure modes, forecasting tools, and domain-specific agents.<n>We evaluate this problem on AssetOpsBench (AOB), an industrial agent benchmark whose plan-execute pipeline exposes repeated overhead from tool discovery, LLM planning, MCP tool execution, and final summarization.<n>We propose two complementary optimization layers for AOB plan-execute pipelines: a temporal semantic cache and a set of MCP workflow optimizations combining disk-backed tool-discovery caching and dependency-aware parallel step execution.
  arXiv  Detail & Related papers  (2026-05-20T02:30:07Z)
• SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters [13.891522069967507]
  We propose a shift to program-level scheduling for compound AI workloads.<n>We present SAGA, a distributed scheduler that implements this abstraction through three mechanisms.
  arXiv  Detail & Related papers  (2026-05-01T09:05:28Z)
• Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scaling [4.857151634248649]
  Large language models are increasingly deployed as complex agentic systems that scale with task complexity.<n>At the algorithm level, allocating additional inference-time can enhance workflow capacity but introduces cross-path redundancy.<n>At the task level, complex tasks can be decomposed into subproblems and delegated across multiple agents for improved scalability and parallelism.<n>We propose Hive, a multi-agent infrastructure that enables algorithm- and task-level scaling.
  arXiv  Detail & Related papers  (2026-04-19T09:59:35Z)
• Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs [62.17306142810532]
  We present Chimera, a predictive scheduling system for multi-agent workflow serving on heterogeneous LLM clusters.<n>Chime traces the best latency, reducing end-to-end latency by 1.2--2.4$times$ and improving task performance by 8.0-9.5 percentage points on average over competitive baselines.
  arXiv  Detail & Related papers  (2026-03-23T17:01:42Z)
• Efficient Multimodal Planning Agent for Visual Question-Answering [67.26245301307539]
  This paper proposes a method that trains a multimodal planning agent, dynamically decomposing the mRAG pipeline to solve the VQA task.<n>In our experiments, the agent can help reduce redundant computations, cutting search time by over 60% compared to existing methods.
  arXiv  Detail & Related papers  (2026-01-28T14:58:59Z)
• Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks [1.2292307778008844]
  We present a comprehensive evaluation of prompt caching across three major Large Language Model (LLM) providers.<n>Our results demonstrate that prompt caching reduces API costs by 45-80% and improves time to first token by 13-31% across providers.
  arXiv  Detail & Related papers  (2026-01-09T18:41:57Z)
• SimpleMem: Efficient Lifelong Memory for LLM Agents [73.74399447715052]
  We introduce SimpleMem, an efficient memory framework based on semantic lossless compression.<n>We propose a three-stage pipeline designed to maximize information density and token utilization.<n> Experiments on benchmark datasets show that our method consistently outperforms baseline approaches in accuracy, retrieval efficiency, and inference cost.
  arXiv  Detail & Related papers  (2026-01-05T21:02:49Z)
• Cache Mechanism for Agent RAG Systems [32.587302983158274]
  ARC (Agent RAG Cache Mechanism) is a novel, annotation-free caching framework that dynamically manages small, high-value corpora for each agent.<n>Our results demonstrate that ARC can drastically enhance efficiency and effectiveness in RAG-powered LLM agents.
  arXiv  Detail & Related papers  (2025-11-04T19:02:29Z)
• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents [84.62985963113245]
  We introduce MEM1, an end-to-end reinforcement learning framework that enables agents to operate with constant memory across long multi-turn tasks.<n>At each turn, MEM1 updates a compact shared internal state that jointly supports memory consolidation and reasoning.<n>We show that MEM1-7B improves performance by 3.5x while reducing memory usage by 3.7x compared to Qwen2.5-14B-Instruct on a 16-objective multi-hop QA task.
  arXiv  Detail & Related papers  (2025-06-18T19:44:46Z)
• Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints [52.58352707495122]
  We present a multi-robot allocation algorithm that decouples the key computational challenges of sequential decision-making under uncertainty and multi-agent coordination. We validate our results over a wide range of simulations on two distinct domains: multi-arm conveyor belt pick-and-place and multi-drone delivery dispatch in a city.
  arXiv  Detail & Related papers  (2020-05-27T01:10:41Z)
• A Non-Stationary Bandit-Learning Approach to Energy-Efficient Femto-Caching with Rateless-Coded Transmission [98.47527781626161]
  We study a resource allocation problem for joint caching and transmission in small cell networks. We then formulate the problem as to select a file from the cache together with a transmission power level for every broadcast round. In contrast to the state-of-the-art research, the proposed approach is especially suitable for networks with time-variant statistical properties.
  arXiv  Detail & Related papers  (2020-04-13T09:07:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.