Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

• URL: http://arxiv.org/abs/2608.03893v1
• Date: Tue, 04 Aug 2026 16:26:47 GMT
• Title: Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse
• Abstract summary: We propose cross-model KV cache transfer, where the receiver reuses the source's KV cache, skipping prefill.<n>We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension.<n>This linear mapper retains 73-98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply.
• Score: 3.9548951874725433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source's KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B->32B, one source layer explains 56% of variance in the target's keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top-k most predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on a small calibration set of 500 FineWeb-Edu sequences of 1,024 tokens each. Surprisingly, across six pairs in three families, this linear mapper retains 73-98% of the receiver's standalone-prefill accuracy on four pairs, while two degrade sharply. A nonlinear MLP recovers up to +37 pp HellaSwag retention on the failures. The mapper runs 2.7-25x faster than re-prefill and remains stable across multi-turn handoff, making cross-model KV cache transfer practical.

Related papers

• CacheBridge: Efficient Cross-Model KV Cache Transfer [12.058564378845373]
  Full-Head Mapping maps each target KV head from every source KV head in selected layers, making transfer quality sensitive to architectural differences.<n>We introduce CacheBridge, which co-designs architecture-indexed mapper support, attention-aligned calibration, and bounded mapper construction.<n>On Qwen3 $14mathrmBto32mathrmB$, it reduces mapper storage by $8times$, accelerates application by up to $3.0times$, matches fullhead with one tenth of the calibration data, and reduces 500-sequence construction from 92.63 to 8.
  arXiv  Detail & Related papers  (2026-09-01T08:20:35Z)
• A Universal Context-Reuse Layer for Cross-Model KV Sharing [7.914652203965336]
  We study emphcross-model KV sharing, which translates the KV state produced by a source model into a representation that can be consumed by a different target model.<n>For Qwen2.5-7B $rightarrow$ Qwen2.5-1.5B, translated KV states improve LongBench2 accuracy from 27.59% to 34.48%, a gain of 6.89 percentage points over the native 1.5B baseline.<n>For the cross-family Qwen2.5-1.5B $rightarrow$ Gemma-2-2B setting, KV handoff reduces target-side prefill cost by
  arXiv  Detail & Related papers  (2026-08-31T15:28:17Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• SemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference [3.2938220347051175]
  We show that uniform KV quantization on a fractional-bit grid does not degrade gracefully.<n>We show that replacing the affine base with a distortion-optimized quantizer (TurboQuant-MSE) prunes the cliff in every protocol tested.
  arXiv  Detail & Related papers  (2026-08-28T22:19:47Z)
• IntentKV: Cross-Turn Intent-Aware KV Cache Pruning for Agent Inference [12.04455190856202]
  Multi-turn LLM agents fan short queries into long trajectories of tool calls, search results, and intermediate reasoning.<n>We introduce IntentKV, learned KV pruning that keeps the base LLM frozen.<n>IntentKV matches the no-pruning full-cache baseline with almost no accuracy drop under tight KV budgets.
  arXiv  Detail & Related papers  (2026-06-06T15:54:48Z)
• CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM [8.82843279804122]
  We introduce CONF-KV, a KV-cache manager that chooses the per-step cache budget.<n> tokens are ranked by a composite of accumulated attention mass and recency.<n>On 75 VisualWebArena tasks it retains 95.3% of full- KV success at 2.8 times lower peak memory.
  arXiv  Detail & Related papers  (2026-05-24T00:15:34Z)
• Linear Predictability of Attention Heads in Large Language Models [0.5833117322405447]
  We show that pretrained Transformers exhibit a pervasive inter-head linear structure.<n>We exploit this by caching only reference-head KV states and reconstructing the remaining heads on the fly.
  arXiv  Detail & Related papers  (2026-03-04T09:04:03Z)
• ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection [8.362927764080203]
  We propose a training-free scorer that ranks tokens by Euclidean distance to the key centroid.<n>ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression.<n> WindowedManifoldKV restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff.
  arXiv  Detail & Related papers  (2026-02-09T07:28:55Z)
• Reconstructing KV Caches with Cross-layer Fusion For Enhanced Transformers [35.286226181391754]
  Cross-layer KV Cache sharing offers a path to mitigate KV Cache bottleneck, but it typically underperforms within-layer methods like GQA.<n>We propose FusedKV, whose top-layer KV caches are a learnable fusion of the most informative ones from the bottom and middle layers.<n>Compared to FusedKV, FusedKV-Lite reduces I/O overhead at the cost of a slight increase in perplexity.
  arXiv  Detail & Related papers  (2025-12-03T15:22:00Z)
• Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads [47.05385031325841]
  SkipV1Former is a Transformer variant that uses skip connections from the first layer's Value heads to strengthen representation and reduce KV cache.<n>We show that SkipV1Former delivers consistent reductions of approximately 25 % in KV cache.<n>When combined with YOCO, it cuts KV cache size by nearly 50 % while still improving performance.
  arXiv  Detail & Related papers  (2025-10-19T12:17:42Z)
• CommonKV: Compressing KV Cache with Cross-layer Parameter Sharing [54.34080239841088]
  CommonKV is a training-free method for cross-layer KV cache compression through adjacent parameters sharing.<n>We show that the proposed method consistently outperforms existing low-rank and cross-layer approaches at various compression ratios.
  arXiv  Detail & Related papers  (2025-08-22T06:55:45Z)
• KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding [72.12756830560217]
  Large language models (LLMs) based on Transformer Decoders have become the preferred choice for conversational generative AI.<n>Despite the overall superiority of the Decoder architecture, the gradually increasing Key-Value cache during inference has emerged as a primary efficiency bottleneck.<n>By down-sampling the Key-Value vector dimensions into a latent space, we can significantly reduce the KV Cache footprint and improve inference speed.
  arXiv  Detail & Related papers  (2025-07-15T12:52:12Z)
• More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression [71.42818367729573]
  In large language models (LLMs), the memory usage of KV cache has become a critical bottleneck during inference.<n>The mainstream KV compression methods, including KV pruning and KV quantization, primarily focus on either token or precision dimension separately.<n>In this paper, we comprehensively investigate the token-precision trade-off in KV cache compression.
  arXiv  Detail & Related papers  (2024-12-17T09:20:31Z)
• KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization [67.74400574357472]
  LLMs are seeing growing use for applications which require large context windows, and with these large context windows KV cache activations surface as the dominant contributor to memory consumption during inference.<n> Quantization is a promising approach for compressing KV cache activations; however, existing solutions fail to represent activations accurately in sub-4-bit precision.<n>Our work, KVQuant, facilitates low precision KV cache quantization by incorporating several novel methods.
  arXiv  Detail & Related papers  (2024-01-31T18:58:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.