LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents

• URL: http://arxiv.org/abs/2602.01053v1
• Date: Sun, 01 Feb 2026 06:36:46 GMT
• Title: LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents
• Abstract summary: We propose LRAgent, a KV cache sharing framework for multi-LoRA agents.<n> LRAgent decomposes the cache into a shared base component from the pretrained weights and an adapter-dependent component from LoRA weights.<n> LRAgent achieves throughput and time-to-first-token latency close to fully shared caching.
• Score: 9.162948089580143
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Role specialization in multi-LLM agent systems is often realized via multi-LoRA, where agents share a pretrained backbone and differ only through lightweight adapters. Despite sharing base model weights, each agent independently builds and stores its own KV cache for the same long, tool-augmented trajectories, incurring substantial memory and compute overhead. Existing KV cache sharing methods largely overlook this multi-LoRA setting. We observe that, across agents, cache differences are dominated by adapter outputs, while activations from the shared pretrained backbone remain highly similar. Based on this observation, we propose LRAgent, a KV cache sharing framework for multi-LoRA agents that decomposes the cache into a shared base component from the pretrained weights and an adapter-dependent component from LoRA weights. LRAgent reduces memory overhead by sharing the base component and storing the adapter component in its inherent low-rank form, and further reduces compute overhead, enabled by shared-$A$ multi-LoRA architectures, by also sharing the low-rank cache and avoiding redundant computations for contexts already processed by other agents. To efficiently reconstruct adapter contributions at runtime, we introduce Flash-LoRA-Attention, a kernel that reorders attention computation to avoid materializing the low-rank cache to full dimension. LRAgent achieves throughput and time-to-first-token latency close to fully shared caching, while preserving accuracy near the non-shared caching baseline across agentic question-answering benchmarks.

Related papers

• Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach [58.97285859000893]
  We study whether $A$ should be shared across clients while $B$ remains client-specific (Share-A/Local-B) or whether $B$ should instead be shared while $A$ remains client-specific (Share-B/Local-A)<n>We propose Federated Adaptive Factor Sharing Low-Rank Adaptation (FedAS-LoRA), which selects the sharing side before training to enhance fine-tuning performance.
  arXiv  Detail & Related papers  (2026-08-10T15:38:45Z)
• Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models [7.174865411448373]
  Federated fine-tuning of foundation models using Low-Rank Adaptation (LoRA) offers a communication efficient solution for distributed learning.<n>Existing federated LoRA methods suffer from two fundamental limitations: (1) structural aggregation bias, and (2) client side lag, as clients repeatedly re LoRA parameters across communication rounds, slowing convergence.<n>We propose HyperLoRA, a unified framework that addresses both issues through amortized federated adaptation through hypernetwork-driven LoRA generation and product space aggregation.
  arXiv  Detail & Related papers  (2026-06-04T13:28:48Z)
• ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache [8.323914332215598]
  Low-Rank Adaptation (LoRA) introduces a critical memory footprint bottleneck during serving.<n>By exploiting the structural properties of LoRA, ForkKV physically decouples the KV cache into a massive shared component.<n>We show that ForkKV achieves up to 3.0x the throughput of state-of-the-art multi-LoRA serving systems.
  arXiv  Detail & Related papers  (2026-04-07T18:52:25Z)
• tLoRA: Efficient Multi-LoRA Training with Elastic Shared Super-Models [8.42285475305854]
  tLoRA is a framework that enables efficient batch training of multiple LoRA jobs.<n> Evaluations using real-world cluster traces demonstrate that tLoRA improves training by 1.2--1.8x, job training completion time by 2.3--5.4x, and GPU utilization by 37%.
  arXiv  Detail & Related papers  (2026-02-06T23:26:02Z)
• Faster Than SVD, Smarter Than SGD: The OPLoRA Alternating Update [50.36542772932594]
  Low-Rank Adaptation (LoRA) fine-tunes large models by learning low-rank updates on top of frozen weights.<n>There is still a gap between full training with low-rank projections (SVDLoRA) and LoRA fine-tuning, indicating that LoRA steps can be further improved.
  arXiv  Detail & Related papers  (2025-09-24T10:32:50Z)
• RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory [57.449129198822476]
  RCR is a role-aware context routing framework for multi-agent large language model (LLM) systems.<n>It dynamically selects semantically relevant memory subsets for each agent based on its role and task stage.<n>A lightweight scoring policy guides memory selection, and agent outputs are integrated into a shared memory store.
  arXiv  Detail & Related papers  (2025-08-06T21:59:34Z)
• Tensorized Clustered LoRA Merging for Multi-Task Interference [29.488565631016332]
  In multi-task settings, merging LoRA adapters trained on heterogeneous sources frequently causes textittask interference, degrading downstream performance.<n>We propose a tensorized clustered LoRA library targeting to address the task interference.<n> TC-LoRA achieves +1.4% accuracy on Phi-3 and +2.3% on Mistral-7B, demonstrating the effectiveness of TC-LoRA in LLM adaptation.
  arXiv  Detail & Related papers  (2025-08-06T01:26:43Z)
• HoneyBee: Efficient Role-based Access Control for Vector Databases via Dynamic Partitioning [7.712955046857144]
  HoneyBee is a dynamic partitioning framework that bridges the gap between current approaches and Role-Based Access Control (RBAC) policies.<n>HoneyBee produces overlapping partitions where vectors can be strategically replicated across different partitions to reduce query latency.<n>HoneyBee formulates the partitioning strategy as a constrained optimization problem to dynamically balance storage, query efficiency, and recall.
  arXiv  Detail & Related papers  (2025-05-02T18:59:31Z)
• Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management [13.327937177980969]
  Multiple Low-Rank Adapters (Multi-LoRAs) are gaining popularity for task-specific Large Language Model (LLM) applications.<n>Existing Multi-LoRA inference systems fail to optimize serving performance like Time-To-First-Toke (TTFT)<n>FASTLIBRA comprises a dependency-aware cache manager and a performance-driven cache swapper.
  arXiv  Detail & Related papers  (2025-04-19T13:17:34Z)
• In-Context Meta LoRA Generation [61.690065588534296]
  Low-rank Adaptation (LoRA) has demonstrated remarkable capabilities for task specific fine-tuning.<n>We propose In-Context Meta LoRA (ICM-LoRA), a novel approach that efficiently achieves task-specific customization of large language models.<n>ICM-LoRA enables more accurate LoRA parameter reconstruction than current parameter reconstruction methods.
  arXiv  Detail & Related papers  (2025-01-29T13:12:01Z)
• Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learning [57.36978335727009]
  Low-Rank Adaptation (LoRA) offers an efficient way to fine-tune large language models (LLMs) In this paper, we propose a framework that adaptively retrieves and composes multiple LoRAs based on input prompts.
  arXiv  Detail & Related papers  (2024-06-24T05:24:41Z)
• S-LoRA: Serving Thousands of Concurrent LoRA Adapters [59.490751234925206]
  Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning method, is often employed to adapt a base model to a multitude of tasks. We present S-LoRA, a system designed for the scalable serving of many LoRA adapters.
  arXiv  Detail & Related papers  (2023-11-06T17:26:17Z)
• CA-LoRA: Adapting Existing LoRA for Compressed LLMs to Enable Efficient Multi-Tasking on Personal Devices [78.16679232748196]
  We introduce a Compression-Aware LoRA (CA-LoRA) framework to transfer Large Language Models (LLMs) to other tasks. Experiment results demonstrate that CA-LoRA outperforms the vanilla LoRA methods applied to a compressed LLM. The source code of CA-LoRA is available at https://github.com/thunlp/CA-LoRA.
  arXiv  Detail & Related papers  (2023-07-15T04:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.