CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG

• URL: http://arxiv.org/abs/2608.07458v1
• Date: Fri, 07 Aug 2026 17:51:49 GMT
• Title: CoinRAG: Contextualized Information Nugget KV Cache Reuse for Long-Context RAG
• Abstract summary: CoinRAG (Contextualized Information KV Cache Reuse for Long-Context RAG) is proposed.<n>Instead of full-chunk encoding, CoinRAG identifies query-relevant semantic units within retrieved chunks through two-stage retrieval.<n>Extensive evaluations on LongBench question answering tasks demonstrate that CoinRAG significantly reduces operational costs and outperforms the other baselines.
• Score: 6.614653387089263
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent optimization studies on Retrieval-Augmented Generation (RAG) have exploited chunk-level KV cache reuse to avoid processing long retrieved contexts for higher efficiency, while significant information redundancy and noise still remain in the coarse-grained chunks. This paper optimizes the Pareto frontier under low prefill latency constraints while maximizing accuracy by proposing CoinRAG (Contextualized Information Nugget KV Cache Reuse for Long-Context RAG). The name metaphorically reflects our core mechanism: much like assembling small tokens (or "coins") to accumulate a larger value, CoinRAG compositionally reuses offline-computed, fine-grained nugget caches to form a learned contextual representation efficiently in a more semantically relevant but compact manner. Specifically, instead of full-chunk encoding, CoinRAG identifies query-relevant semantic units within retrieved chunks through two-stage retrieval and seamlessly assembles their sliced KV representations with a chunk-level context. Extensive evaluations on LongBench multi-hop question answering tasks demonstrate that CoinRAG significantly reduces operational costs and outperforms the other baselines with a new Pareto frontier and an average 5.3% relative improvement in answer quality (F1) under a standard fast prefill latency budget.

Related papers

• BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference [20.58039191111583]
  Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation.<n>The resulting key-value ( KV) cache grows linearly with sequence length and creates severe memory bottlenecks.<n>We propose BeaconKV, a training-free KV cache compression method that maintains beacon queries, compact representatives for each global query cluster.
  arXiv  Detail & Related papers  (2026-09-04T10:23:16Z)
• CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration [67.17709877327232]
  Cate KV is a hybrid KV cache method that retains only critical token information for consistent heads.<n>Cate KV reduces memory usage by up to $2.72times$ and accelerates decoding by $2.18times$ in single-sample inputs.
  arXiv  Detail & Related papers  (2026-08-31T06:02:37Z)
• No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval [51.43543998583709]
  Single-stage Sparse Retrieval (SSR) is a paradigm shift that replaces expensive clustering with efficient sparse coding.<n>It reduces indexing time by 15x compared to ColBERTv2, halves retrieval latency, and simultaneously improves retrieval performance.
  arXiv  Detail & Related papers  (2026-05-28T15:53:34Z)
• Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction [65.710271475739]
  We introduce a global retention-based KV eviction method that learns each token's future utility under a unified memory budget.<n>We show that our method substantially reduces KV memory while matching or surpassing full-cache inference.<n>These results suggest that learned, globally calibrated KV eviction is not only a compression technique, but also a mechanism for improving long-context reasoning.
  arXiv  Detail & Related papers  (2026-05-10T16:47:50Z)
• Kwai Summary Attention Technical Report [69.40814939510126]
  Long-context ability, has become one of the most important direction of next-generation Large Language Models.<n>Standard softmax attention exhibits quadratic time complexity with respect to sequence length.<n>We propose Kwai Summary Attention (KSA), a novel attention mechanism that reduces sequence modeling cost by compressing historical contexts.
  arXiv  Detail & Related papers  (2026-04-27T12:59:53Z)
• $A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving [42.02864241423696]
  Large language models (LLMs) have demonstrated strong capabilities in processing long contexts.<n>Despite their ability to handle long sequences, the resulting decoding latency and memory overhead remain substantial.<n>Recent advances in KV Cache reuse have shown potential to mitigate these costs, but still suffer from notable performance degradation.
  arXiv  Detail & Related papers  (2025-11-13T07:28:59Z)
• REFRAG: Rethinking RAG based Decoding [67.4862300145604]
  REFRAG is an efficient decoding framework that compresses, senses, and expands to improve latency in RAG applications.<n>We provide rigorous validation of REFRAG across diverse long-context tasks, including RAG, multi-turn conversations, and long document summarization.
  arXiv  Detail & Related papers  (2025-09-01T03:31:44Z)
• SCBench: A KV Cache-Centric Analysis of Long-Context Methods [61.025422435235456]
  We introduce SCBench, a benchmark for evaluating long-context methods from a KV cachecentric perspective.<n>We provide an extensive KV cache-centric analysis of eight categories long-context solutions, including Gated Linear RNNs and Mamba-Attention hybrids.<n>Our findings show that sub-O(n) memory methods suffer in multi-turn scenarios, while sparse encoding with O(n) memory and sub-O(n2) pre-filling perform robustly.
  arXiv  Detail & Related papers  (2024-12-13T17:59:52Z)
• ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression [10.003118268356017]
  Long context poses significant challenges for inference efficiency.<n>We introduce ClusterKV, which recalls tokens at the granularity of semantic clusters.<n>Experiment results show that ClusterKV attains negligible accuracy loss across various tasks with 32k context lengths.
  arXiv  Detail & Related papers  (2024-12-04T10:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.