LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

• URL: http://arxiv.org/abs/2608.11231v1
• Date: Fri, 31 Jul 2026 02:10:44 GMT
• Title: LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs
• Abstract summary: We present LinearKV, a training-free hybrid-PIC framework.<n>We find that a emphsingle cached state suffices as the linear layer's initializer.<n>We compare the two across three hybrid models and three PIC selectors.
• Score: 43.666302638883025
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC). Existing PIC methods, however, are built for full-attention models, where a token-indexed KV cache underlies its core operations: matching reusable token chunks, concatenating their KV entries, and selectively recomputing a few tokens to restore cross-chunk context. Hybrid LLMs break these primitives---they replace most attention layers with linear recurrences that expose only a fixed-size state, leaving no token-indexed KV to concatenate or to locally repair. This raises a natural question: can PIC benefit hybrid models, and what would it take? We present LinearKV, a training-free hybrid-PIC framework. Its key insight is a \emph{decoupled initialization}: each linear layer maps its $K$ matched local states to a single initial state, while full-attention layers concatenate their KV as before. LinearKV is therefore compatible with existing PIC methods, reusing their token selection and recomputation as-is. Under this framework, we find that a \emph{single cached state} suffices as the linear layer's initializer. The algebraically principled alternative---composing all $K$ cached states into the exact full-prefix state, as concurrent work HYPIC does---is unnecessary and, on some architectures, even harmful. We compare the two across three hybrid models and three PIC selectors. On the two GDN models the two tie, both recovering most of full quality (up to $92\%$); on the Mamba-2 model, exact composition instead collapses under every selector---under EPIC, for instance, it recovers only $46.6\%$ of full quality, versus $86.8\%$ for a single cached block initializer. A single state initializer is also cheaper, cutting time-to-first-token to $0.46\times$ full prefill versus a further $5$--$17\%$ overhead for exact composition; results hold across LongBench QA and RULER at 8K--32K.

Related papers

• Flash-CNNCap: Capacitance Extraction via Image Mapping [3.790585344640331]
  Flash-CNNCap is a CNN-based extractor that reformulates full-matrix capacitance prediction as image-to-image regression over spatial contribution maps.<n>A total-capacitance model and a master-conditioned coupling model each predict a spatial map that is reduced to conductor-level values.<n>The maps are learned from conductor-level labels without per-pixel supervision.
  arXiv  Detail & Related papers  (2026-07-26T22:46:20Z)
• Regularize or Localize: When Training-Time KV-Cache Geometry Pays Under Quantization [7.954260884574662]
  sigreg reduces hidden-state pairwise-cosine anisotropy by $38%$ across three paired seeds.<n>Applying sigreg directly to K and V during continued training, however, reduces mean cache anisotropy by $94%$ across four checkpoints.
  arXiv  Detail & Related papers  (2026-07-19T01:13:09Z)
• Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes [70.61868608402723]
  We introduce $textbfSelf-Correcting Coupled Markov Jump Processes (SC-CMJP)<n>In conjunction with SC-CMJP, we introduce $textttCO_texttt2textttJump$, a novel training-free single-pass sampler for joint multimodal geneneration.<n>For training and evaluation purposes, we have created and will release three large-scale joint multimodal generation corpora.
  arXiv  Detail & Related papers  (2026-07-14T18:39:29Z)
• You Only Index Once: Cross-Layer Sparse Attention with Shared Routing [61.29627714699688]
  Cross-layer sparse attention (A) is built on top of KV-sharing architectures such as YOCO.<n>A single indexer computes token-level top-k selection once and reuses the resulting index across layers.<n>Results show CLSA is both accurate and efficient, achieving up to 7.6x decoding speedup and 17.1x overall throughput improvement at 128K context.
  arXiv  Detail & Related papers  (2026-06-04T17:54:04Z)
• SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation [0.0]
  Large Language Models generate text autoregressively, relying on a key-value (KV) cache whose memory footprint grows linearly with context length.<n>Recent compression methods mitigate this cost via token merging.<n>We propose a training-free, dual-component framework for KV cache compression that addresses these limitations.
  arXiv  Detail & Related papers  (2026-05-19T17:44:56Z)
• DualKV: Shared-Prompt Flash Attention for Efficient RL Training with Large Rollouts and Long Contexts [32.57334288518084]
  Modern RL post-training methods such as GRPO and DAPO train on $N$ response sequences of $R$ tokens sampled from a shared prompt.<n>Standard FlashAttention replicates all $P$ prompt tokens $N times across both forward and backward passes.<n>We propose textDualAttention, the first Flash kernel variant eliminates shared-prompt replication during training.
  arXiv  Detail & Related papers  (2026-05-14T21:11:32Z)
• HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference [23.016627145448467]
  Multimodal Large Language Models (MLLMs) have advanced unified reasoning over text, images, and videos.<n>Each visual input expands into thousands of tokens, causing caches to scale linearly with context length.<n>A common solution is to compress caches under a fixed allocated budget at different granularities.<n>We propose HybridKV, a hybrid KV cache compression framework that integrates complementary strategies in three stages.
  arXiv  Detail & Related papers  (2026-04-07T13:51:07Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• Canonicalizing Multimodal Contrastive Representation Learning [76.15228959754727]
  We show that across model families such as CLIP, SigLIP, and FLAVA, a geometric relationship exists between their embedding spaces.<n>This finding enables backward-compatible model upgrades, avoiding costly re-embedding, and has implications for the privacy of learned representations.
  arXiv  Detail & Related papers  (2026-02-19T18:09:36Z)
• CODA: Repurposing Continuous VAEs for Discrete Tokenization [31.932323809073477]
  textbfCODA(textbfCOntinuous-to-textbfDiscrete textbfAdaptation) is a framework that decouples compression and discretization.<n>Our approach achieves a remarkable codebook utilization of 100% and notable reconstruction FID (rFID) of $mathbf0.43$ and $mathbf1.34$ for $8 times$ and $16 times$ compression on ImageNet 256$times$ 256 benchmark.
  arXiv  Detail & Related papers  (2025-03-22T12:59:00Z)
• Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference [78.65321721142624]
  We focus on a memory bottleneck imposed by the key-value ( KV) cache. Existing KV cache methods approach this problem by pruning or evicting large swaths of relatively less important KV pairs. We propose LESS, a simple integration of a constant sized cache with eviction-based cache methods.
  arXiv  Detail & Related papers  (2024-02-14T18:54:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.