TriAttention: Efficient Long Reasoning with Trigonometric KV Compression

• URL: http://arxiv.org/abs/2604.04921v1
• Date: Mon, 06 Apr 2026 17:58:42 GMT
• Title: TriAttention: Efficient Long Reasoning with Trigonometric KV Compression
• Abstract summary: Extended reasoning in large language models (LLMs) creates severe KV cache memory bottlenecks.<n>Leading KV cache compression methods estimate KV importance using attention scores from recent post-RoPE queries.<n>We propose TriAttention to estimate key importance by leveraging these centers.
• Score: 42.4775416331056
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Extended reasoning in large language models (LLMs) creates severe KV cache memory bottlenecks. Leading KV cache compression methods estimate KV importance using attention scores from recent post-RoPE queries. However, queries rotate with position during RoPE, making representative queries very few, leading to poor top-key selection and unstable reasoning. To avoid this issue, we turn to the pre-RoPE space, where we observe that Q and K vectors are highly concentrated around fixed non-zero centers and remain stable across positions -- Q/K concentration. We show that this concentration causes queries to preferentially attend to keys at specific distances (e.g., nearest keys), with the centers determining which distances are preferred via a trigonometric series. Based on this, we propose TriAttention to estimate key importance by leveraging these centers. Via the trigonometric series, we use the distance preference characterized by these centers to score keys according to their positions, and also leverage Q/K norms as an additional signal for importance estimation. On AIME25 with 32K-token generation, TriAttention matches Full Attention reasoning accuracy while achieving 2.5x higher throughput or 10.7x KV memory reduction, whereas leading baselines achieve only about half the accuracy at the same efficiency. TriAttention enables OpenClaw deployment on a single consumer GPU, where long context would otherwise cause out-of-memory with Full Attention.

Related papers

• AgentKV: Phase-Aware KV Eviction for Agentic LLMs [15.992100550192765]
  KV-eviction methods score cached keys against representative queries drawn from the most recent tokens.<n>We show that future queries form a mixture over think, act, tool, and others phases, and principal-angle analysis shows these components occupy measurably different query subspaces.<n>We propose AGENTKV, which maintains a small query buffer per phase and scores cached keys against their union.
  arXiv  Detail & Related papers  (2026-09-14T00:42:53Z)
• CateKV: On Sequential Consistency for Long-Context LLM Inference Acceleration [67.17709877327232]
  Cate KV is a hybrid KV cache method that retains only critical token information for consistent heads.<n>Cate KV reduces memory usage by up to $2.72times$ and accelerates decoding by $2.18times$ in single-sample inputs.
  arXiv  Detail & Related papers  (2026-08-31T06:02:37Z)
• Information-Aware KV Cache Compression for Long Reasoning [71.77807044404499]
  Existing KV cache compression methods mainly rely on attention weights to estimate token importance.<n>We propose textbfInfoKV, an entropy-aware KV cache compression framework that incorporates information-theoretic signals.
  arXiv  Detail & Related papers  (2026-06-25T11:03:37Z)
• QUOKA: Query-Oriented KV Selection For Efficient LLM Prefill [5.014026212750645]
  We present QUOKA: Query-oriented KV selection for efficient attention.<n>We show that QUOKA achieves near-baseline accuracy, utilizing 88% fewer key-value pairs per attention evaluation.
  arXiv  Detail & Related papers  (2026-02-09T14:32:26Z)
• Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference [54.467557491325046]
  We propose Pre-hoc Sparsity (PrHS), which selects KV entries before attention scoring and provides explicit accuracy control.<n>PrHS reduces retrieval overhead by over 90%, achieving 3x higher retrieval sparsity than HShare at matched or better accuracy.<n>It incurs under 1% average degradation on LongBench, lowers attention FLOPs by about 15% versus prior baselines, and yields a 9.9x speedup in attention-operator latency and 2.8x higher throughput.
  arXiv  Detail & Related papers  (2026-02-09T07:05:23Z)
• CTkvr: KV Cache Retrieval for Long-Context LLMs via Centroid then Token Indexing [28.184704036272787]
  Long contexts pose significant challenges for inference efficiency in large language models.<n>We propose CTKVR, a novel centroid-then-token KV retrieval scheme.<n>CTKVR achieves superior performance across multiple benchmarks with less than 1% accuracy degradation.
  arXiv  Detail & Related papers  (2025-12-17T15:56:32Z)
• G-KV: Decoding-Time KV Cache Eviction with Global Attention [57.47409249054187]
  Large language models (LLMs) excel in complex tasks but encounter significant computational and memory challenges due to long sequence lengths.<n> KV cache compression has emerged as an effective approach to greatly enhance the efficiency of reasoning.<n>We propose G-KV, a KV cache eviction method that employs a global scoring mechanism, combining local and historical attention scores to more accurately assess token importance.
  arXiv  Detail & Related papers  (2025-11-29T14:21:33Z)
• Value-Guided KV Compression for LLMs via Approximated CUR Decomposition [24.262712463465665]
  CurDKV is a novel, value-centric KV compression method that selects keys and values based on leverage scores computed from CUR matrix decomposition.<n>Our approach approximates the dominant subspace of the attention output $softmax(QKT)V$, ensuring that the retained tokens best preserve the model's predictive behavior.
  arXiv  Detail & Related papers  (2025-09-18T15:04:06Z)
• Homogeneous Keys, Heterogeneous Values: Exploiting Local KV Cache Asymmetry for Long-Context LLMs [28.554617623874176]
  We show a fundamental yet previously overlooked asymmetry in KV caches.<n>Key-value asymmetry reveals a critical limitation in existing compression methods.<n>We propose a training-free compression framework (AsymKV) that combines homogeneity-based key merging with a mathematically proven lossless value compression.
  arXiv  Detail & Related papers  (2025-06-04T16:10:44Z)
• More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression [71.42818367729573]
  In large language models (LLMs), the memory usage of KV cache has become a critical bottleneck during inference.<n>The mainstream KV compression methods, including KV pruning and KV quantization, primarily focus on either token or precision dimension separately.<n>In this paper, we comprehensively investigate the token-precision trade-off in KV cache compression.
  arXiv  Detail & Related papers  (2024-12-17T09:20:31Z)
• Squeezed Attention: Accelerating Long Context Length LLM Inference [61.787865959140994]
  We propose Squeezed Attention to accelerate applications where a large portion of the input context is fixed.<n>During inference, we compare query tokens from the user input with the centroids to predict which keys from the fixed context are semantically relevant.<n>We also present a hierarchical version of our algorithm which can reduce the complexity of attention from linear to logarithmic with respect to the fixed context length.
  arXiv  Detail & Related papers  (2024-11-14T18:54:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.