Dual-Cache Latent Space Communication between Heterogeneous Language Models

• URL: http://arxiv.org/abs/2608.20617v1
• Date: Thu, 20 Aug 2026 23:35:36 GMT
• Title: Dual-Cache Latent Space Communication between Heterogeneous Language Models
• Abstract summary: Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context.<n>Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's.<n>XKV is 26% faster than LCF-X and 6.8x faster than text communication.
• Score: 11.183066364515094
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multi-agent LLM systems split work across models, so answering often requires knowledge that sits in another agent's context: a Sharer has encoded information that a Receiver needs to complete its task. They usually communicate by exchanging text, which puts autoregressive decoding on the critical path and reduces the exchange to a discrete message written without sight of the receiver's state. Recent latent protocols instead translate the sharer's key-value (KV) cache into the receiver's: C2C supports heterogeneous models but requires both to read the same input, while LCF-X removes this shared-context requirement through position-free sharer-cache pooling. Three restrictions remain: LCF-X compresses the sharer alone, supplies the same layer-local summary to every receiver position with no joint cross-layer memory to retrieve from, and assumes matched layer count and KV geometry. We introduce XKV, which lifts all three: learned-query attention pools both caches; self-attention over receiver-aligned layer tokens, with a learned layer map reconciling different depths, mixes the pooled summaries into a compact joint memory; and a shared position decoder lets every raw receiver cache position retrieve its own per-head-gated residual in the receiver's native KV geometry. Both models stay frozen and may differ in family, depth, KV-head count, head dimension, and tokenizer; only the translator is trained. Across 45 dataset-model-pair settings (six heterogeneous and three same-model ordered pairings, five datasets), XKV attains the highest macro score and best average rank, improving on LCF-X on every dataset (by 4.6 exact-match and 4.2 F1 points on ROPES) and surpassing text communication on four of the five, while training 76% fewer parameters and translating a cache pair 10.3x faster (5.8 vs. 59.9 ms); end to end, XKV is 26% faster than LCF-X and 6.8x faster than text communication.

Related papers

• Interface-Aware KV Cache Quantization for Dense On-Chip NVM in Long-Context LLM Decoding [5.984068900226547]
  Key-value (KV) cache is the dominant memory bottleneck in long-context large language model (LLM) decoding.<n>Existing KV quantization methods were designed for GPU-style memory systems.<n>This paper examines what these structures cost when the KV cache resides in NVM behind fixed-range converters.
  arXiv  Detail & Related papers  (2026-09-04T22:58:21Z)
• CacheBridge: Efficient Cross-Model KV Cache Transfer [12.058564378845373]
  Full-Head Mapping maps each target KV head from every source KV head in selected layers, making transfer quality sensitive to architectural differences.<n>We introduce CacheBridge, which co-designs architecture-indexed mapper support, attention-aligned calibration, and bounded mapper construction.<n>On Qwen3 $14mathrmBto32mathrmB$, it reduces mapper storage by $8times$, accelerates application by up to $3.0times$, matches fullhead with one tenth of the calibration data, and reduces 500-sequence construction from 92.63 to 8.
  arXiv  Detail & Related papers  (2026-09-01T08:20:35Z)
• XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication [41.12003897083975]
  Heterogeneous multi-agent LLM systems can outperform homogeneous configurations by reducing redundant reasoning patterns.<n>Existing communication protocols either operate through text, discarding the sender's internal representations, or require architectural homogeneity for latent-level transfer.<n>We identify the entity grounding problem in cross-architecture communication: cross-attention bridges that transfer continuous representations across different LLM families suffer from rare-token compression collapse.<n>We propose XBRIDGE, a decode-free communication protocol that addresses this through two mechanisms.
  arXiv  Detail & Related papers  (2026-08-12T05:32:33Z)
• StereoFactory: A Unified Merging Framework for Robust Stereo Matching [61.973843344605655]
  Stereo matching has advanced through foundation models trained on large-scale datasets, yet this paradigm suffers from a scalability bottleneck.<n>Model merging offers a scalable post-hoc alternative by integrating knowledge from specialized models after source checkpoints are available.<n>We propose StereoFactory, a coarse-to-fine evolutionary framework for adaptive model merging.
  arXiv  Detail & Related papers  (2026-06-16T03:36:59Z)
• Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights [58.27172968382275]
  Multi-agent LLM systems usually collaborate by exchanging natural-language messages.<n>We study an alternative communication interface: instead of appending a sender's message to the receiver's context, compile the sender's hidden states into a transient, receiver-specific weight perturbation.
  arXiv  Detail & Related papers  (2026-05-13T17:58:32Z)
• PackCache: A Training-Free Acceleration Method for Unified Autoregressive Video Generation via Compact KV-Cache [61.57938553036056]
  We introduce PackCache, a training-free KV-cache management method that compacts the KV cache through three coordinated mechanisms.<n>In terms of efficiency, PackCache accelerates end-to-end generation by 1.7-2.2x on 48-frame long sequences.
  arXiv  Detail & Related papers  (2026-01-07T19:51:06Z)
• Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compression [0.0]
  We introduce Q-KVComm, a new protocol that enables direct transmission of compressed key-value (KV) cache representations between agents.<n>Q-KVComm achieves 5-6x compression ratios while maintaining semantic fidelity, with coherence quality scores above 0.77 across all scenarios.<n>Our work establishes a new paradigm for LLM agent communication, shifting from text-based to representation-based information exchange.
  arXiv  Detail & Related papers  (2025-11-27T10:45:41Z)
• KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse [30.48395228595732]
  We describe KVLink, an approach for efficient key-value ( KV) cache reuse in large language models (LLMs)<n> KVLink introduces two key techniques: adjusting positional embeddings of the KV cache at inference to match the global position after concatenation, and using trainable special tokens to restore self-attention.<n> Experiments across 7 datasets demonstrate that KVLink improves question answering accuracy by an average of 4% over state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-02-21T23:34:29Z)
• QuantSpec: Self-Speculative Decoding with Hierarchical Quantized KV Cache [67.84112700032007]
  Large Language Models (LLMs) are increasingly being deployed on edge devices for long-context settings.<n>In these scenarios, the Key-Value ( KV) cache is the primary bottleneck in terms of both GPU memory and latency.<n>We propose a novel self-speculative decoding framework, QuantSpec, where the draft model shares the architecture of the target model but employs a hierarchical 4-bit quantized KV cache and 4-bit quantized weights for acceleration.
  arXiv  Detail & Related papers  (2025-02-05T20:43:48Z)
• LoCoCo: Dropping In Convolutions for Long Context Compression [77.26610232994508]
  This paper presents a novel approach, Dropping In Convolutions for Long Context Compression (LoCoCo) LoCoCo employs only a fixed-size Key-Value ( KV) cache, and can enhance efficiency in both inference and fine-tuning stages.
  arXiv  Detail & Related papers  (2024-06-08T01:35:11Z)
• Joint Beam Search Integrating CTC, Attention, and Transducer Decoders [53.297697898510194]
  We propose a joint modeling scheme where four decoders share the same encoder.<n>The 4D model is trained jointly, which will bring model regularization and maximize the model robustness.<n>In addition, we propose three novel joint beam search algorithms by combining three decoders.
  arXiv  Detail & Related papers  (2024-06-05T05:18:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.