Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel

• URL: http://arxiv.org/abs/2607.14431v1
• Date: Wed, 15 Jul 2026 23:55:48 GMT
• Title: Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel
• Abstract summary: We make a frozen small language model both more capable and dramatically cheaper at once, without changing any weights.<n>We show that own-position graft is the numerically exact operating point on a model with floating-point rotary encoding.<n>We describe the system at the behavior level; the engine is proprietary, and every reported number is backed by committed input and output so the scoring can be re-checked without it.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We report a way to make a frozen small language model both more capable and dramatically cheaper at once, without changing any weights. Verified knowledge is deposited once as a byte-exact key-value (KV) state artifact and later restored, by graft, into a fresh inference context. The restore is bit-exact: under a pinned deterministic configuration, the grafted logits are byte-for-byte identical to a fresh computation (SHA-256 equality), with zero KL divergence and 100% argmax agreement over fifty samples. We show that own-position graft is the unique numerically exact operating point on a model with floating-point rotary encoding, and we verify byte-exactness on two model scales (12B, 31B) and two GPU targets, one through a pre-registered replay. On AIME 2025, a frozen Gemma-4-12B moves from 80.0% to 93.3% once a verified solution library is grafted, above its own 77.5% and its 31B sibling's 89.2% published anchors. On the recurring case, eight problems the base model never solves within a 401,026-token budget are answered from cached verified solutions in 61 total decode tokens, a factor of 6,574 fewer tokens and about 8,700x less energy; the capability claim proper rests on held-out transfer (7 of 7 at 31B). The same byte-exact store widens usable context from 32,768 to 2,854,766 tokens at zero extra accelerator memory, and moves byte-identical between machines of the same architecture. We describe the system at the behavior level; the engine is proprietary, and every reported number is backed by committed input and output hashes so the scoring can be re-checked without it.

Related papers

• Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models [67.72077328700088]
  We present the first large scale study of overtraining decoder-only dense transformer models varying two dimensions simultaneously.<n>We find that Token-1B models outperform byte models (End-Of-Token-1B and Bytes-1B) in the low-FLOP regime but eventually plateau.<n>By operating over a small vocabulary of 256 bytes instead of on the order of 100K tokens, they circumvent the need for top-kruncation during logit dumping.
  arXiv  Detail & Related papers  (2026-09-11T00:17:22Z)
• Invalidation Contracts for Cross-Episode Agent Memory [0.8673353590830692]
  LLM agents that cache recovery suggestions from API errors can skip re-derivation in later episodes.<n>Server-side data drift turns those cached fixes into silent failures.<n>We introduce invalidation contracts, a protocol layer that attaches version stamps and cacheability hints to every recovery suggestion.
  arXiv  Detail & Related papers  (2026-08-31T18:45:07Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• Paritok-4B: Intent-Conditioned Context Compression for Coding Agents [0.9689471757238836]
  Paritok-4B is a 4B LoRA compressor for coding-agent trajectories built on two commitments.<n>It selects spans rather than rewriting them, and 96.4% of the identifiers, paths, and numbers it emits already appear in its input.<n>It is intent-conditioned: told the agent's current task, it acts chiefly inside a retained segment, selecting which lines survive.
  arXiv  Detail & Related papers  (2026-08-25T07:53:30Z)
• A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever [0.0]
  We show how to solve language problems with a persistent memory of verified solutions.<n>Every new instance of a problem family is answered at zero generation tokens, bit-exact, deterministically.<n>On published benchmarks, frontier models remain far ahead of any 12B at raw from-scratch reasoning.
  arXiv  Detail & Related papers  (2026-07-26T19:08:33Z)
• Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware [0.0]
  Single-stream autoregressive decoding of large language models is bound by memory bandwidth.<n>A small draft model proposes $K$ tokens autoregressively, the target model scores all of them in one batched pass.<n>We present a from-scratch, device-agnostic (CUDA/MPS/CPU) implementation and an empirical study across five draft/target backend configurations.
  arXiv  Detail & Related papers  (2026-07-19T15:01:35Z)
• Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning [0.16921396880325776]
  We introduce Support Vector Attention (SV-Attention), a memory whose weights are support coefficients of a one-class Sfp with fixed box parameter C.<n>Its active-set partition gives reserve tokens exactly zero weight, output weights; a incremental incremental solver token to recover the state produced by retraining without it under the same C.<n>We also demonstrate surgical eviction, exact editing, patient-record deletion, and a forgettable retrieval memory over real sentence embeddings.
  arXiv  Detail & Related papers  (2026-07-13T23:13:57Z)
• Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata [57.27430779838529]
  We present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton.<n>The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding.<n> Empirical evaluations on Dream-7B and LLaDA-8 show substantial accuracy gains across various tasks.
  arXiv  Detail & Related papers  (2026-07-08T05:48:57Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• RoPE-Aware Bit Allocation for KV-Cache Quantization [52.099459337231345]
  Block-GTQ is a bit allocator for key-cache quantization built on TurboQuant-MSE.<n>It better preserves RoPE query-key logits on a ten-model diagnostic panel.<n>It runs 1.34x faster than fp16 FlashAttention2 at 128K context.
  arXiv  Detail & Related papers  (2026-06-23T00:17:48Z)
• A Verifiable Search Is Not a Learnable Chain-of-Thought [0.0]
  It is tempting to assume any task solvable by a short program can be taught to a model as its chain-of-thought.<n>This paper shows the assumption fails for an identifiable class of procedures.
  arXiv  Detail & Related papers  (2026-06-20T04:52:14Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• StateSMix: Online Lossless Compression via Mamba State Space Models and Sparse N-gram Context Mixing [0.0]
  StateSMix is an online-trained Mamba-style State Space Model (SSM) with sparse n-gram context mixing and arithmetic coding.<n>The SSM provides a continuously-updated probability estimate over BPE tokens, while nine sparse n-gram hash tables add exact local and long-range pattern memorisation.<n>On the standard enwik8 benchmark, StateSMix achieves 2.123 bpb on 1 MB, 2.149 bpb on 3 MB, and 2.162 bpb on 10 MB, beating xz -9e (LZMA2) by 8.7%, 5.4%, and
  arXiv  Detail & Related papers  (2026-04-05T19:28:11Z)
• Learn by Surprise, Commit by Proof [0.0]
  We propose a self-gated post-training framework for autonomous knowledge acquisition.<n>When a passage produces anomalously high pertoken loss, L SCP flags it, generates a Q&A chain that forces the model to articulate its own knowledge.<n>The entire learning intensity is governed by a single parameter $r$.
  arXiv  Detail & Related papers  (2026-04-02T12:17:10Z)
• The Residual Stream Is All You Need: On the Redundancy of the KV Cache in Transformer Inference [3.378773775514883]
  Key-value (KV) cache is widely treated as essential state in transformer inference.<n>We prove that keys and values at every layer are deterministic projections of the residual stream.<n>We build on this result with KV-Direct, a bounded-memory inference scheme.
  arXiv  Detail & Related papers  (2026-03-20T05:59:50Z)
• The case for 4-bit precision: k-bit Inference Scaling Laws [75.4335600212427]
  Quantization methods reduce the number of bits required to represent each parameter in a model. The final model size depends on both the number of parameters of the original model and the rate of compression. We run more than 35,000 zero-shot experiments with 16-bit inputs and k-bit parameters to examine which quantization methods improve scaling for 3 to 8-bit precision.
  arXiv  Detail & Related papers  (2022-12-19T18:48:33Z)
• HAWQV3: Dyadic Neural Network Quantization [73.11579145354801]
  Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values. We present HAWQV3, a novel mixed-precision integer-only quantization framework.
  arXiv  Detail & Related papers  (2020-11-20T23:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.