WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization

• URL: http://arxiv.org/abs/2607.28699v2
• Date: Thu, 06 Aug 2026 10:44:15 GMT
• Title: WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
• Abstract summary: KV-cache quantization is validated today by offline benchmark averages.<n>A deployed system cannot tell whether compression is damaging the request it is serving right now.<n>We give it a provably sound runtime meter -- a "DTrace for KV quantization"
• Score: 20.158719055939383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: KV-cache quantization is validated today by offline benchmark averages; a deployed system cannot tell whether compression is damaging the request it is serving right now. We give it a provably sound runtime meter -- a "DTrace for KV quantization": a per-(layer, head, step) upper bound on the total variation between exact and compressed attention. The meter has two tiers: a deterministic band-norm-witness bound, sound for any cache-preserving black-box quantizer and for any query (adaptive-safe, worst-case Cauchy--Schwarz plus RoPE band-unitarity), and a tighter probabilistic certificate for a controlled subtractively-dithered INT8 quantizer under an explicit request-level failure budget (stated for non-adaptive queries; core theorems machine-checked in Lean 4). Three results. Observability: the meter enters SGLang through an env-guarded patch, and any scheme registered as one tensor function is measured in live serving. Repair: meter-driven gating -- risk-ranked where the witness is saturated, certified where it is informative -- empirically restores the quality floor at benchmark scale, e.g. raw-cast fp8 from 22.8 back to 79.7 on hard RULER tasks with the difference from uncompressed bounded at $[+0.0,+0.8]$ by a paired test. Analysis: aggressive schemes survive on cross-layer error cancellation, not per-step fidelity -- in a 28-layer sweep, no single layer's pollution alone loses anything (0/28) -- and the certified int8 cache serves $1.88\times$ more KV tokens at the same memory in SGLang. All artifacts, guards, and the Lean development are released at https://github.com/metask-ai/witcert-kv-certificates; every number regenerates from the shipped artifacts by one command.

Related papers

• SemKV: Semantic Mixed-Precision KV Cache Quantization Guided by the Quality Cliff for Long-Context LLM Inference [3.2938220347051175]
  We show that uniform KV quantization on a fractional-bit grid does not degrade gracefully.<n>We show that replacing the affine base with a distortion-optimized quantizer (TurboQuant-MSE) prunes the cliff in every protocol tested.
  arXiv  Detail & Related papers  (2026-08-28T22:19:47Z)
• Data Diversity, Not Frequency Invariance: A Controlled and Self-Audited Study of Compression-Robust Deepfake Detection [0.26856688022781555]
  Frequency features and compression-invariant representation learning are widely assumed to be key to deepfake detection that survives video compression.<n>We test this with CAFRL - block-DCT and FFT-phase streams, compression-level-conditioned band attention, and adversarial (gradient-reversal) compression.<n>Under a pre-registered protocol with capacity- and augmentation-matched controls, a plain EfficientNet-B0 on multi-quality data beat CAFRL as specified at every compression level.
  arXiv  Detail & Related papers  (2026-08-26T12:05:10Z)
• HaloMark: A Spectral Threshold for Embedding-Vector Watermarking under C2PA [3.0458514384586404]
  We present HaloMark, a watermark for embedding vectors cryptographically bound to a C2PA manifest.<n>The verifier runs at 284 us and 24 bytes of sidecar per vector, validated end-to-end against three C2PA reference-SDK bindings.
  arXiv  Detail & Related papers  (2026-08-09T11:36:59Z)
• Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression [42.94199567901975]
  KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that supports it.<n>We test this assumption for large reasoning models and show that it can fail.<n>We evaluate ten token-eviction KV compression methods and one quantization method on three models across mathematical reasoning, scientific QA, clinical calculation, and long-context retrieval.
  arXiv  Detail & Related papers  (2026-08-03T03:03:53Z)
• Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution [0.0]
  Practitioners accept a compressed language model once it clears a stack of data-cheap quality guards.<n>Across three model families, gently-compressed models clear every guard and then invent procedure steps that were never in the instructions.<n>We characterize the blindspot and dissociation with paired confidence intervals on a pre-registered, powered canary.
  arXiv  Detail & Related papers  (2026-07-30T13:33:09Z)
• RoPE-Aware Bit Allocation for KV-Cache Quantization [52.099459337231345]
  Block-GTQ is a bit allocator for key-cache quantization built on TurboQuant-MSE.<n>It better preserves RoPE query-key logits on a ten-model diagnostic panel.<n>It runs 1.34x faster than fp16 FlashAttention2 at 128K context.
  arXiv  Detail & Related papers  (2026-06-23T00:17:48Z)
• Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation [6.129872931808218]
  Key-value ( KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory.<n>In this study, we explore alignment preservation under KV cache quantization.<n>We find that low-bit quantization can silently destroy safety alignment.
  arXiv  Detail & Related papers  (2026-06-01T02:02:20Z)
• Runtime-Certified Bounded-Error Quantized Attention [0.0]
  We present a tiered KV cache architecture that enables runtime-certified attention.<n>A two-term error decomposition yields per-head, per-step bounds on (i) attention distribution distortion from key quantization and (ii) value reconstruction error.<n>Across PG-19, NIAH, and RULER benchmarks, the system matches dense FP16 KV quality within noise for language modelling and retrieval tasks.
  arXiv  Detail & Related papers  (2026-05-20T08:04:40Z)
• Echo: KV-Cache-Free Associative Recall with Spectral Koopman Operators [45.88028371034407]
  KV-cache-free associative recall architecture built around Spectral Koopman Attention (SKA)<n>We introduce Echo, a KV-cache-free associative recall architecture built around Spectral Koopman Attention (SKA)
  arXiv  Detail & Related papers  (2026-05-07T22:26:27Z)
• The Illusion of Equivalence: Systematic FP16 Divergence in KV-Cached Autoregressive Inference [2.954681536906518]
  KV caching is a ubiquitous optimization in autoregressive transformer inference.<n>We show that FP16 KV cache inference is fundamentally non-equivalent to recomputation.
  arXiv  Detail & Related papers  (2026-04-16T15:59:40Z)
• Prune-Quantize-Distill: An Ordered Pipeline for Efficient Neural Network Compression [4.049313299965171]
  Motivated by this gap between compression and acceleration, we study a practical, ordered pipeline that targets measured latency.<n>We combine three widely used techniques: unstructured pruning, quantization-aware training (QAT), and knowledge distillation (KD)<n>Across all settings, the ordered pipeline achieves a stronger accuracy-size-latency frontier than any single technique alone.
  arXiv  Detail & Related papers  (2026-04-05T06:13:47Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill and Decode Inference [48.40143137402824]
  Multi-Head Latent Attention (MLA) compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory.<n>In tensor parallelism (TP), however, attention heads are computed across multiple devices, and each device must load the full cache.<n>We propose-Parallel Latent Attention (TPLA), a scheme that partitions both the latent representation and each head's input dimension across devices, performs attention independently per shard, and then combines results with an all-reduce.
  arXiv  Detail & Related papers  (2025-08-21T15:25:40Z)
• vCache: Verified Semantic Prompt Caching [95.16654660556975]
  This paper proposes vCache, the first verified semantic cache with user-defined error rate guarantees.<n>It employs an online learning algorithm to estimate an optimal threshold for each cached prompt, enabling reliable cache responses without additional training.<n>Our experiments show that vCache consistently meets the specified error bounds while outperforming state-of-the-art static-threshold and fine-tuned embedding baselines.
  arXiv  Detail & Related papers  (2025-02-06T04:16:20Z)
• SoftPatch+: Fully Unsupervised Anomaly Classification and Segmentation [84.07909405887696]
  This paper is the first to consider fully unsupervised industrial anomaly detection (i.e., unsupervised AD with noisy data)<n>We propose memory-based unsupervised AD methods, SoftPatch and SoftPatch+, which efficiently denoise the data at the patch level.<n>Compared with existing methods, SoftPatch maintains a strong modeling ability of normal data and alleviates the overconfidence problem in coreset.<n> Comprehensive experiments conducted in diverse noise scenarios demonstrate that both SoftPatch and SoftPatch+ outperform the state-of-the-art AD methods on the MVTecAD, ViSA, and BTAD benchmarks.
  arXiv  Detail & Related papers  (2024-12-30T11:16:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.