A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever

• URL: http://arxiv.org/abs/2607.23806v1
• Date: Sun, 26 Jul 2026 19:08:33 GMT
• Title: A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever
• Abstract summary: We show how to solve language problems with a persistent memory of verified solutions.<n>Every new instance of a problem family is answered at zero generation tokens, bit-exact, deterministically.<n>On published benchmarks, frontier models remain far ahead of any 12B at raw from-scratch reasoning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Improving a language model today means retraining it: enormous compute, a new opaque model each cycle, non-deterministic output. We take the opposite path: the model stays frozen, and a persistent memory of verified solutions grows beside it. Once a problem family is solved and has passed an independent verification step that never consults the answer key, every new instance of that family is answered at zero generation tokens, bit-exact, deterministically. Across 180 fresh instances spanning nine problem families, four architectures from four vendors - dense and mixture-of-experts - each score 180/180 at zero generation tokens per answer: execution-bound capability decoupled from parameter scaling. A negative control attributes the capability fully to the memory: emptied, it solves nothing. The same verify-before-store contract holds for open-ended reasoning: 88/88 consistency-gated acceptances across all four models, machine-checked formal proof, and reasoning-method transfer at 77/80. Memory selection takes 1.4 microseconds; a full reuse completes in 6-23 ms at 36 mWh. Approximate similarity retrieval selects the wrong item 94.3% of the time on a 4,500-item verified store where exact addressing makes zero errors. The store also serves as working context at a scale no shipped engine matches: a 6,000,000-token movable window on a single 46 GB GPU at flat memory, where vLLM stops at 30,399 tokens and SGLang silently truncates past 32,000. On published benchmarks, frontier models remain far ahead of any 12B at raw from-scratch reasoning; on everything this system has solved and verified, the comparison inverts: a frontier API call pays a fresh generation pass on every query, forever, while verified reuse costs zero tokens and returns the identical bits every time. A public testbench with free, rate-limited access accompanies this report: https://corbenic-galahad-bench.hf.space

Related papers

• NovaFabric: Tamper-Evident, Replayable Evidence for Autonomous AI Agent Runs [0.0]
  We present NovaFabric, producing audit-grade execution evidence.<n>It records an agent run, without modifying agent logic, into a portable Run Capsule.<n>Sealed runs are re-executable under a four-mode replay protocol.<n>We evaluate eight research questions at measured scope.
  arXiv  Detail & Related papers  (2026-09-11T08:35:43Z)
• Interface-Induced Trajectory Censoring [6.496285418517832]
  We report a tool-call rate read off the serving stack.<n>That number can be zero while the model is emitting well-formed calls.<n>We reproduce the funnel across a 21x scale range of Qwen2.5-Coder.
  arXiv  Detail & Related papers  (2026-09-03T15:00:31Z)
• Turning Interaction History into Execution State: A Runtime Layer for Long-Horizon Coding Agents [48.967927222079965]
  Ledger is a runtime layer that distills an agent's completed interactions into an explicit execution state.<n> Ledger keeps this state in an online execution ledger and applies it at two boundaries of every step.<n>Across all 500 SWE-bench Verified instances, Ledger raises Pass@1 from 56.2% to 64.2% with GPT-5 mini and from 75.8% to 81.0% with MiniMax M2.5.
  arXiv  Detail & Related papers  (2026-08-01T18:25:00Z)
• Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models [0.0]
  Two experiments cross all three factors on one held, contamination-free synthetic corpus.<n>Experiment 1 measures instruction-following decay as rule count N grows from 10 to 160, crossed with four formats and system-prompt vs. user-turn placement.<n>Experiment 2 measures recall accuracy, false-premise sycophancy, and absent-fact fabrication across a 2k-to-512k-token context ladder.
  arXiv  Detail & Related papers  (2026-07-21T16:31:35Z)
• A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi [0.0]
  This report keeps the hardware and asks what a model that fits can do.<n>We deploy Mini-V-4.6, a modern multimodal assistant pairing a SigLIP2 encoder and window-attention merger.<n>The system answers an image question end-to-end in 1.7s.
  arXiv  Detail & Related papers  (2026-07-16T04:48:44Z)
• Smarter and Cheaper at Once: Byte-Exact KV-Cache Grafting Turns a Frozen Small Model into a Verified-Knowledge Flywheel [0.0]
  We make a frozen small language model both more capable and dramatically cheaper at once, without changing any weights.<n>We show that own-position graft is the numerically exact operating point on a model with floating-point rotary encoding.<n>We describe the system at the behavior level; the engine is proprietary, and every reported number is backed by committed input and output so the scoring can be re-checked without it.
  arXiv  Detail & Related papers  (2026-07-15T23:55:48Z)
• Auto: The AGI Compiler [0.0]
  Auto is a compiler that records live agent behavior, measures which parts are secretly deterministic, extracts them into verified programs or distilled specialists, and emits cognition binaries.<n>We use "AGI compiler" in one narrow, testable sense: a system that autonomously converts novel experience into permanent, verified, near-free skill while measuring what it does not know.
  arXiv  Detail & Related papers  (2026-07-05T23:09:24Z)
• Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation [0.0]
  Per-token counterfactual credit estimation asks which token in a language-model rollout caused the final answer to be right or wrong.<n>We measure what that assumption costs on a stock inference engine, with a three-pass design.<n>Across six configurations and three models, re-feeding changes the credit estimate at rates 14-28 percentage points above the replica floor.
  arXiv  Detail & Related papers  (2026-06-14T06:09:16Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification [35.08680804423239]
  Chain-of-thought (CoT) reasoning improves large language models (LLMs) on difficult tasks, but it also makes inference expensive.<n>We propose Latent-Then-Explicit Reasoning (LaTER), a two-stage paradigm that first performs bounded exploration in a continuous latent space.<n>LaTER projects final-layer hidden states back to the input embedding space, preserves the latent KV cache, and uses entropy and model-native stop-token probes to decide when to switch.
  arXiv  Detail & Related papers  (2026-05-08T06:23:58Z)
• DyePack: Provably Flagging Test Set Contamination in LLMs Using Backdoors [52.85182605005619]
  We introduce DyePack, a framework that leverages backdoor attacks to identify models that used benchmark test sets during training.<n>Like how banks mix dye packs with their money to mark robbers, DyePack mixes backdoor samples with the test data to flag models that trained on it.<n>We evaluate DyePack on five models across three datasets, covering both multiple-choice and open-ended generation tasks.
  arXiv  Detail & Related papers  (2025-05-29T02:22:14Z)
• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach [70.44265766483633]
  We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space.<n>Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time.<n>We show that the resulting model can improve its performance on reasoning benchmarks, sometimes dramatically.
  arXiv  Detail & Related papers  (2025-02-07T18:55:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.