The Detection-Extraction Gap: Models Know the Answer Before They Can Say It

• URL: http://arxiv.org/abs/2604.06613v2
• Date: Thu, 09 Apr 2026 17:43:28 GMT
• Title: The Detection-Extraction Gap: Models Know the Answer Before They Can Say It
• Abstract summary: We show that chain-of-thought tokens are produced after the answer is recoverable from a partial prefix.<n>Free continuations from early prefixes recover the correct answer even at 10% of the trace, while forced extraction fails on 42% of these cases.<n>We propose Black-box Adaptive Early Exit (BAEE), which uses free continuations for both detection and extraction.
• Score: 1.423262366328995
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern reasoning models continue generating long after the answer is already determined. Across five model configurations, two families, and three benchmarks, we find that 52--88% of chain-of-thought tokens are produced after the answer is recoverable from a partial prefix. This post-commitment generation reveals a structural phenomenon: the detection-extraction gap. Free continuations from early prefixes recover the correct answer even at 10% of the trace, while forced extraction fails on 42% of these cases. The answer is recoverable from the model state, yet prompt-conditioned decoding fails to extract it. We formalize this mismatch via a total-variation bound between free and forced continuation distributions, yielding quantitative estimates of suffix-induced shift. Exploiting this asymmetry, we propose Black-box Adaptive Early Exit (BAEE), which uses free continuations for both detection and extraction, truncating 70--78% of serial generation while improving accuracy by 1--5pp across all models. For thinking-mode models, early exit prevents post-commitment overwriting, yielding gains of up to 5.8pp; a cost-optimized variant achieves 68--73% reduction at a median of 9 API calls. Code is available at https://github.com/EdWangLoDaSc/know2say.

Related papers

• ResiSpec: Enhancing Multi-Candidate Speculative Sampling via Residual Distribution Shaping [79.1905186547363]
  ResiSpec is a framework that strategically reforms the proposal distribution during verification.<n>It prevents candidate obsolescence and achieves up to 1.92$times$ speedup over state-of-the-art multi-candidate methods.
  arXiv  Detail & Related papers  (2026-08-25T11:25:55Z)
• TelemetrySuffBench: Is Agent Telemetry Sufficient for Failure-Origin Diagnosis? [1.7077233340294056]
  We introduce TelemetrySuffBench, a benchmark that separates failure detection, fault-origin localization, and safe abstention under insufficient evidence.<n>We evaluate five frontier language models using unified protocols, explicit candidate sets, invalid-output accounting, subgroup analyses, and a frozen blind holdout.
  arXiv  Detail & Related papers  (2026-08-08T03:54:22Z)
• Token Budget Saturation and Mechanistic Early Detection of Reasoning Non-Convergence in Chain-of-Thought Models [1.4563514298965277]
  Chain-of-thought reasoning models exhibit a bimodal convergence pattern.<n>Generations terminate within a token budget (converged) or exhaust it without reaching a conclusion (non-converged)<n>We show that converged generations achieve 90.3% accuracy on AIME 1983-2024 while non-converged ones achieve only 6.6%, with an overall convergence rate of 62.0%.
  arXiv  Detail & Related papers  (2026-07-23T15:37:04Z)
• Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs [52.149036302760386]
  Large language models can be fine-tuned to encode prompt-borne secrets into fluent, seemingly benign outputs.<n>Recent work proposes detection using linear probes that recover the secret from internal activations.<n>We show that this defense can be systematically evaded, but that detectability can be recovered through a targeted data-level intervention.
  arXiv  Detail & Related papers  (2026-06-08T12:27:11Z)
• Off-the-Shelf LLMs as Process Scorers: Training-Free Alternative to PRMs for Mathematical Reasoning [51.88950852117154]
  Chunk-Level Guided Generation is a training-free alternative that uses an off-the-shelf large language model as a process scorer.<n>We show that scoring variable-length reasoning steps with large-model likelihoods is unreliable due to a systematic length bias.<n>Chunk-Level Guided Generation produces substantially shorter reasoning traces than PRM guided search.
  arXiv  Detail & Related papers  (2026-06-01T04:43:36Z)
• ESPO: Early-Stopping Proximal Policy Optimization [78.79610718910628]
  ESPO (Early-Stopping Proximal Policy Optimization) detects trajectory failure on-the-fly and terminates rollouts early.<n>On DeepSeek-R1-Distill-Qwen-7B trained for mathematical reasoning, ESPO surpasses PPO on AIME2024 (46.28% vs. 45.25%), AMC2023 (85.83% vs. 82.94%), and MATH-500 (87.42% vs. 85.43%)
  arXiv  Detail & Related papers  (2026-05-28T12:40:22Z)
• LaTER: Efficient Test-Time Reasoning via Latent Exploration and Explicit Verification [35.08680804423239]
  Chain-of-thought (CoT) reasoning improves large language models (LLMs) on difficult tasks, but it also makes inference expensive.<n>We propose Latent-Then-Explicit Reasoning (LaTER), a two-stage paradigm that first performs bounded exploration in a continuous latent space.<n>LaTER projects final-layer hidden states back to the input embedding space, preserves the latent KV cache, and uses entropy and model-native stop-token probes to decide when to switch.
  arXiv  Detail & Related papers  (2026-05-08T06:23:58Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• Broken Chains: The Cost of Incomplete Reasoning in LLMs [2.8220022980767054]
  Reasoning-specialized models like OpenAI's 5.1 and DeepSeek-V3.2 allocate substantial inference compute to extended chain-of-thought traces.<n>We introduce a framework that constrains models to reason exclusively through code, comments, both, or neither, then systematically ablates token budgets to 10%, 30%, 50%, and 70% of optimal.
  arXiv  Detail & Related papers  (2026-02-16T03:57:51Z)
• MERGETUNE: Continued fine-tuning of vision-language models [77.8627788911249]
  Fine-tuning vision-language models (VLMs) often leads to catastrophic forgetting of pretrained knowledge.<n>We introduce a novel paradigm, continued fine-tuning (CFT), which seeks to recover pretrained knowledge after a zero-shot model has already been adapted.
  arXiv  Detail & Related papers  (2026-01-15T15:15:53Z)
• Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production [55.76222360698305]
  We explore a class of supervised training objectives that allow a language model to dynamically and autonomously scale the number of compute steps used for each input token.<n>For any token, the model can request additional compute steps by emitting a don't know> output.<n>We find that the CYB model requests additional steps when doing so improves accuracy, and the model adapts its processing time to token-level complexity and context.
  arXiv  Detail & Related papers  (2025-10-13T21:07:05Z)
• Valid Stopping for LLM Generation via Empirical Dynamic Formal Lift [6.908972852063454]
  Sequential-EDFL reduces generation by 22-28% vs. sequential baselines.<n>EDFL serves as a first-stage filter reducing verification burden by 83%.
  arXiv  Detail & Related papers  (2025-10-07T21:28:53Z)
• Rainbow Padding: Mitigating Early Termination in Instruction-Tuned Diffusion LLMs [10.214443153276962]
  Diffusion large language models exhibit a critical vulnerability we term textteos> overflow.<n>We introduce Rainbow Padding, a simple remedy that replaces repeated textteos> placeholders with a repeating cycle of distinct padding tokens.<n>Experiments show that Rainbow Padding substantially improves length robustness and output quality, with as few as seven padding tokens sufficient to prevent early termination.
  arXiv  Detail & Related papers  (2025-10-04T05:24:27Z)
• Language Models Can Predict Their Own Behavior [29.566208688211876]
  Language models (LMs) can exhibit specific behaviors,' such as a failure to follow alignment training, that we hope to detect and react to during deployment.<n>We show that probes trained on the internal representation of input tokens alone can predict a wide range of eventual behaviors over the entire output sequence.<n>An early warning system built on the probes reduces jailbreaking by 91%.
  arXiv  Detail & Related papers  (2025-02-18T23:13:16Z)
• Language Model Inversion [77.22715643068284]
  We show that next-token probabilities contain a surprising amount of information about the preceding text. Our inversion method reconstructs prompts with a BLEU of $59$ and token-level F1 of $78$ and recovers $27%$ of prompts exactly.
  arXiv  Detail & Related papers  (2023-11-22T19:04:04Z)
• Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation [56.57532238195446]
  We propose a method named Ethicist for targeted training data extraction. To elicit memorization, we tune soft prompt embeddings while keeping the model fixed. We show that Ethicist significantly improves the extraction performance on a recently proposed public benchmark.
  arXiv  Detail & Related papers  (2023-07-10T08:03:41Z)
• Look-back Decoding for Open-Ended Text Generation [62.53302138266465]
  We propose Look-back, an improved decoding algorithm that tracks the distribution distance between current and historical decoding steps. Look-back can automatically predict potential repetitive phrase and topic drift, and remove tokens that may cause the failure modes. We perform decoding experiments on document continuation and story generation, and demonstrate that Look-back is able to generate more fluent and coherent text.
  arXiv  Detail & Related papers  (2023-05-22T20:42:37Z)
• Preventing Posterior Collapse with Levenshtein Variational Autoencoder [61.30283661804425]
  We propose to replace the evidence lower bound (ELBO) with a new objective which is simple to optimize and prevents posterior collapse. We show that Levenstein VAE produces more informative latent representations than alternative approaches to preventing posterior collapse.
  arXiv  Detail & Related papers  (2020-04-30T13:27:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.