TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

• URL: http://arxiv.org/abs/2608.03276v1
• Date: Tue, 04 Aug 2026 07:51:37 GMT
• Title: TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
• Abstract summary: We introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction.<n>Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide.<n>We show that TaskPress efficiently creates a compact, reusable cache across diverse queries.
• Score: 39.9021319826802
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Long-context inference with large language models is constrained by the linear growth of the key-value cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a meta-query during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.

Related papers

• Back from the Future: Key-Value Cache Management by Counter-Causal Surprise [67.1056509495879]
  Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.<n>We propose a simple yet effective KV eviction scheme motivated by the insight that past tokens which can be well-predicted from more recent tokens are redundant.<n>We evaluate our strategy on various open-source LLMs and benchmark datasets showing competitive or improved performance over other state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-30T02:42:51Z)
• Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries [39.38028687042293]
  Key-Value (KV) cache is crucial for efficient Large Language Models (LLMs) inference.<n>Existing KV cache compression methods rely on input-side attention patterns to estimate token importance during the prefill stage.<n>We propose position-aware pseudo queries (DapQ) to approximate KV cache compression via position-aware pseudo queries.
  arXiv  Detail & Related papers  (2026-03-12T05:36:32Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• TableCache: Primary Foreign Key Guided KV Cache Precomputation for Low Latency Text-to-SQL [26.22021341849791]
  We propose precomputing table representations as KV caches offline and querying the required ones online.<n>A key aspect of our approach is the computation of table caches while preserving primary foreign key relationships between tables.
  arXiv  Detail & Related papers  (2026-01-13T17:20:55Z)
• Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks [1.2292307778008844]
  We present a comprehensive evaluation of prompt caching across three major Large Language Model (LLM) providers.<n>Our results demonstrate that prompt caching reduces API costs by 45-80% and improves time to first token by 13-31% across providers.
  arXiv  Detail & Related papers  (2026-01-09T18:41:57Z)
• CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation [7.119276797399788]
  Increasing key-value (KV) cache size poses critical challenges to memory and execution efficiency.<n>Most KV cache compression methods rely on token eviction using all attention heads in Grouped Query Attention (GQA)-based LLMs.<n>We introduce a layer-adaptive KV cache allocation strategy, which consistently outperforms state-of-the-art approaches under various memory budgets.
  arXiv  Detail & Related papers  (2025-08-04T13:26:16Z)
• Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query [48.52389201779425]
  KV cache memory usage grows substantially with longer text sequences.<n>Existing KV cache eviction methods prune tokens using prefilling-stage attention scores.<n>Lookahead Q-Cache generates low-cost pseudo lookahead queries to better approximate the true decoding-stage queries.
  arXiv  Detail & Related papers  (2025-05-24T10:34:38Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• Efficient Inference of Vision Instruction-Following Models with Elastic Cache [76.44955111634545]
  We introduce Elastic Cache, a novel strategy for efficient deployment of instruction-following large vision-language models. We propose an importance-driven cache merging strategy to prune redundancy caches. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation.
  arXiv  Detail & Related papers  (2024-07-25T15:29:05Z)
• User Intent Recognition and Semantic Cache Optimization-Based Query Processing Framework using CFLIS and MGR-LAU [0.0]
  This work analyzed the informational, navigational, and transactional-based intents in queries for enhanced QP. For efficient QP, the data is structured using Epanechnikov Kernel-Ordering Points To Identify the Clustering Structure (EK-OPTICS) The extracted features, detected intents and structured data are inputted to the Multi-head Gated Recurrent Learnable Attention Unit (MGR-LAU)
  arXiv  Detail & Related papers  (2024-06-06T20:28:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.