SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

• URL: http://arxiv.org/abs/2608.03228v2
• Date: Wed, 05 Aug 2026 20:50:42 GMT
• Title: SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval
• Abstract summary: We develop a training free KV cache index that preserves attention scores rather than key reconstruction quality.<n>At rank 32, it removes 13 to 30 percent of PCA's remaining top 64 recall error.<n>It improves 68 to 89 percent of attention heads per model, with the largest gains in deeper layers.
• Score: 4.894648561808929
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing low rank KV cache methods preserve either model weights or key variance, neither of which directly reflects the attention scores used during inference. We derive the expected attention score distortion caused by rank r key compression and show that it yields a covariance weighted low rank objective. Under a margin condition, controlling this distortion also improves top k recall. The optimal rank r solution has a closed form asymmetric factorization obtained from the SVD of the covariance weighted query key operator. This motivates SAKI, a training free KV cache index that directly preserves attention scores rather than key reconstruction quality. Across LLaMA 3.1 8B, Qwen 2.5 7B, Mistral 7B v0.1, and Llama 3.2 3B, SAKI outperforms key PCA at every tested rank. At rank 32, it removes 13 to 30 percent of PCA's remaining top 64 recall error, including improvements from 0.748 to 0.799 on LLaMA 3.1 8B and from 0.786 to 0.850 on Qwen 2.5 7B. It improves 68 to 89 percent of attention heads per model, with the largest gains in deeper layers. Predicted score MSE reductions closely match empirical measurements, with a Pearson correlation of 0.997, while ablation studies confirm that the gains arise from optimizing the attention score objective rather than covariance weighting alone. Analysis of the scoring operator further explains why weight only, invariant subspace, and key reconstruction methods can be suboptimal. SAKI uses random-matrix theory to separate genuine covariance signal from autocorrelated sampling noise, matching PCA with only 512 calibration tokens and adding value exactly where PCA sees no reliable signal.

Related papers

• TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models [40.42873860258522]
  We propose TrustRoboReward, a multi-paradigm reward modeling framework equipped with Preference-Ordered Isotonic Score Editing (POISE)<n>POISE rectifies pointwise scores and eliminates cross-paradigm reversal conflicts unresolved by TrustJudge.<n>Qwen3-VL-4B trained with POISE achieves an overall reward score of 77.96%, nearly matching GPT-5-mini (78.09%, gap 0.13%) and outperforming the strongest RoboReward-4B baseline by 10.13%.
  arXiv  Detail & Related papers  (2026-08-09T05:25:22Z)
• Quality-Diversity Stress Tests for Process Reward Models:What Archive Coverage Can and Cannot Certify [1.0742675209112622]
  We formulate PRM stress testing as a quality-diversity search problem using MAP-Elites.<n>We characterize what such archives certify: finite-cell repair bounds covered-cell tail risk and average residual severity but cannot bound the worst remaining cell from covered fraction alone.<n>A predeclared paired LoRA repair protocol reduces exploit rates from 0.148 to 0.037 to 0.074, the worst attack from 0.333 to 0.177 to 0.212, improves ranking AUROC without degrading best-of-4 accuracy.
  arXiv  Detail & Related papers  (2026-08-08T08:41:06Z)
• Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning [0.16921396880325776]
  We introduce Support Vector Attention (SV-Attention), a memory whose weights are support coefficients of a one-class Sfp with fixed box parameter C.<n>Its active-set partition gives reserve tokens exactly zero weight, output weights; a incremental incremental solver token to recover the state produced by retraining without it under the same C.<n>We also demonstrate surgical eviction, exact editing, patient-record deletion, and a forgettable retrieval memory over real sentence embeddings.
  arXiv  Detail & Related papers  (2026-07-13T23:13:57Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• Rethinking Noise-Robust Training for Frozen Vision Foundation Models: A Cross-Dataset Benchmark with a Case Study of Small-Loss Failure [7.512887610816016]
  Frozen Vision Foundation Models (VFMs) with lightweight classification heads are increasingly used in medical imaging.<n>We present a benchmark of eight noisy-label methods across five medical datasets, three backbones, two noise types, and five noise rates.<n>We conclude with evidence-based guidance and a low-regret feature-space selector for practical recommendation.
  arXiv  Detail & Related papers  (2026-05-21T15:02:44Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Step-wise Rubric Rewards for LLM Reasoning [72.17879367869503]
  Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning in large language models.<n>We find 18.2% of steps in correct responses are wrong yet positively rewarded.<n>Across six mathematical reasoning benchmarks, SRaR improves average accuracy over RaR by 3.57 points.
  arXiv  Detail & Related papers  (2026-05-17T07:08:14Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Pretraining Objective Matters in Extreme Low-Data FGVC: A Backbone-Controlled Study [39.146761527401424]
  We compare four frozen ViT-B/16 encoders trained with supervised classification, contrastive learning (SigLIP2), masked reconstruction (MAE), and self-distillation (DINOv3)<n>Supervised and contrastive encoders provide the strongest linear separability (logistic AUC: 0.768 and 0.735; SVM AUC: 0.739 and 0.697)<n>We find that DINOv3 underperforms across probe families in this domain.
  arXiv  Detail & Related papers  (2026-05-15T04:14:16Z)
• Metric Unreliability in Multimodal Machine Unlearning: A Systematic Analysis and Principled Unified Score [14.579552536669217]
  Unlearning in Vision-Language Models (VLMs) is required for compliance with the General Data Protection Regulation Code.<n>We present the first study of reliability in multimodal unlearning.<n>We introduce the Unified Quality Score (UQS), a metric with weights derived from each metric's Spearman correlation with the metric distance.
  arXiv  Detail & Related papers  (2026-05-04T04:13:00Z)
• Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment [53.72927532626824]
  We show that subsets of just 50 examples (0.3% of data) can achieve over 0.93 Pearson correlation with full benchmark scores.<n>To better predict preferences, we trained regression models on these selected subsets, achieving 0.98 correlation.<n>This demonstrates that in regression modeling, well-curated subsets outpredict the full benchmark, showing quality over quantity.
  arXiv  Detail & Related papers  (2026-04-20T00:57:31Z)
• The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models [0.0]
  Empirically, three independent leaderboards (Open LLM v2, an extended 12-benchmark suite, LiveBench) all have d_eff in [2.86, 4.80] on their competitive frontier.<n>A submodular greedy algorithm with the Nemhauser (1 - 1/e) guarantee finds a stable core of 4 benchmarks.<n>A counterfactual validation across 12 internal benchmarks and 27 categories confirms that the eigenstructure predicts which evaluations are irreplaceable.
  arXiv  Detail & Related papers  (2026-04-15T08:56:58Z)
• CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention [35.44699837487632]
  We propose a Covariance-Aware, Rank-Enhanced MLA conversion pipeline under a fixed KV width.<n> CARE introduces three key steps: (i) activation-preserving factorization, which aligns the approximation with the actual input activations rather than just the weights; (ii) adjusted-rank allocation, which spreads a fixed KV budget across layers by giving more capacity to layers that need it most; and (iii) KV-parity mapping, which re parameterizes the converted K and V to fit the MLA format while keeping the KV-cache size unchanged.
  arXiv  Detail & Related papers  (2026-03-18T17:18:35Z)
• Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inference [54.467557491325046]
  We propose Pre-hoc Sparsity (PrHS), which selects KV entries before attention scoring and provides explicit accuracy control.<n>PrHS reduces retrieval overhead by over 90%, achieving 3x higher retrieval sparsity than HShare at matched or better accuracy.<n>It incurs under 1% average degradation on LongBench, lowers attention FLOPs by about 15% versus prior baselines, and yields a 9.9x speedup in attention-operator latency and 2.8x higher throughput.
  arXiv  Detail & Related papers  (2026-02-09T07:05:23Z)
• Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs [92.7279890407059]
  Multi-head Latent Attention (MLA) is an innovative architecture designed to ensure efficient and economical inference.<n>This paper proposes the first data-efficient fine-tuning method for transitioning from Multi-Head Attention to MLA.
  arXiv  Detail & Related papers  (2025-02-20T18:50:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.