The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies

• URL: http://arxiv.org/abs/2605.10799v2
• Date: Fri, 15 May 2026 04:05:16 GMT
• Title: The Last Word Often Wins: A Format Confound in Chain-of-Thought Corruption Studies
• Abstract summary: Corruption studies infer which steps are computationally important'' from accuracy loss when steps are corrupted.<n>We show that when benchmark chains end with an explicit terminal answer line, these tests largely measure emphanswer placement rather than where intermediate computation is carried out.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Corruption studies, the standard tool for evaluating chain-of-thought (CoT) faithfulness, infer which steps are ``computationally important'' from accuracy loss when steps are corrupted. We show that when benchmark chains end with an explicit terminal answer line, as in GSM8K and MATH, these tests largely measure \emph{answer placement} rather than where intermediate computation is carried out. Using matched GSM8K examples, removing only the final answer statement while preserving all reasoning collapses suffix sensitivity by about $19\times$ for Qwen~2.5-3B ($N{=}300$, $p{=}0.022$). Conflicting-answer prompts, which contain correct reasoning but a wrong explicit final answer, drive accuracy to zero or near-zero at 7B across five open-weight model families; wrong-answer following is strong at 3B--7B and attenuates sharply at larger scales. Replications on MATH, within-stable comparisons at 7B, and suffix-free chains show the same pattern in different guises: corruption sensitivity tracks the location of explicit answer text, not a fixed computational depth in the reasoning. Generation-time probes indicate that final answers are rarely early-determined during generation (${<}5\%$ early commitment), yet consumption-time behavior systematically follows explicit answer text. The confound is therefore largely a readout effect when the chain is consumed. We propose a three-prerequisite protocol (question-only control, format characterization, and an all-position sweep) as a practical minimum for future corruption-based faithfulness studies.

Related papers

• Relational Response Fields: A General Theory of Black-Box LLM Response Consistency and Recovery [0.0]
  Black-box language-model reliability is commonly pursued by sampling, prompting, voting, verifying, or iteratively revising individual answers.<n>We represent responses to typed transformations of a query as a emphrelational response field (RRF)<n>anchors encode how valid responses must change under paraphrase, scaling, decomposition, or other task symmetries.<n>For relation operator $D$, anchor operator $A$, and at most $k$ corrupted response nodes, we identify $_k(D,A)$ as the intrinsic difficulty of black-box response recovery.
  arXiv  Detail & Related papers  (2026-08-05T07:43:41Z)
• Semantic Early-Stopping for Iterative LLM Agent Loops [0.0]
  We study semantic early-stopping: the loop halts when consecutive draft embeddings stop changing in meaning.<n>We generate each question's full trajectory once, replay every stopping policy over the identical drafts, and cache every LLM-judge call.<n>An oracle that selects the best round attains +0.115 Information Score over every practical policy.
  arXiv  Detail & Related papers  (2026-06-25T13:24:21Z)
• When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel [5.106950500256654]
  Chain-of-thought (CoT) traces are increasingly used to improve language model capability and to audit model behavior.<n>We test this assumption with a step-level Detect-Classify-Compare framework built around an answer-commitment proxy.<n>Across nine models and seven reasoning benchmarks, latent commitment and explicit answer arrival align on only 61.9% of steps on average.
  arXiv  Detail & Related papers  (2026-05-12T08:24:47Z)
• Reliable Chain-of-Thought via Prefix Consistency [6.053609157192373]
  We observe that traces with correct answers reproduce their original answer more often than traces with wrong answers.<n>We use this difference as a reliability signal, prefix consistency, that weights each candidate answer by how often it reappears under regeneration.
  arXiv  Detail & Related papers  (2026-05-08T12:28:05Z)
• Rethinking Dense Sequential Chains: Reasoning Language Models Can Extract Answers from Sparse, Order-Shuffling Chain-of-Thoughts [51.84894623128418]
  Modern reasoning language models generate dense, sequential chain-of-thought traces implicitly assuming that every token contributes and that steps must be consumed in order.<n>We challenge both assumptions through a systematic intervention pipeline--removal, masking, shuffling, and noise injection--applied to model-generated reasoning chains.<n>We find that answer extraction operates on a sparse, order-insensitive, and structurally robust informational substrate.
  arXiv  Detail & Related papers  (2026-05-08T06:15:50Z)
• Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes [4.738949927143789]
  We investigate whether linearly decodable failure signals in hidden states can be leveraged to correct those failures.<n>We find that decodable failure structure supports post-generation reliability estimation even when the fixed linear steering family cannot exploit it for correction.
  arXiv  Detail & Related papers  (2026-05-07T05:58:38Z)
• The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimation [1.8345614451086532]
  RLHF-aligned language models exhibit response homogenization on TruthfulQA.<n>40-79% of questions produce a single semantic cluster across 10 i.i.d. samples.
  arXiv  Detail & Related papers  (2026-03-25T09:35:15Z)
• Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thought [0.0]
  We study whether the shape of uncertainty dynamics across reasoning steps--captured by sampling a few answer completions per step--predicts correctness.<n>We introduce entropy-trajectory monotonicity: a chain is monotone if its per-step answer-distribution entropy decreases every step.
  arXiv  Detail & Related papers  (2026-03-19T14:17:16Z)
• Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought [11.955186033088351]
  We provide evidence of performative chain-of-thought (CoT) in reasoning models.<n>We compare activation probing, early forced answering, and a CoT monitor across two large models.<n>We contrast this with genuine reasoning in difficult multihop GPQA-Diamond questions.
  arXiv  Detail & Related papers  (2026-03-05T18:55:16Z)
• How Does Prefix Matter in Reasoning Model Tuning? [57.69882799751655]
  We fine-tune three R1 series models across three core model capabilities: reasoning (mathematics), coding, safety, and factuality.<n>Results show that prefix-conditioned SFT improves both safety and reasoning performance, yielding up to +6% higher Safe@1 accuracy.
  arXiv  Detail & Related papers  (2026-01-04T18:04:23Z)
• TARG: Training-Free Adaptive Retrieval Gating for Efficient RAG [46.122203287541005]
  Training-free Adaptive Retrieval Gating (TARG) is a single-shot policy that decides when to retrieve using only a short, no-latency draft from the base model.<n>On NQ-Open, TriviaQA, and PopQA, TARG consistently shifts the accuracy-efficiency frontier.
  arXiv  Detail & Related papers  (2025-11-12T23:09:52Z)
• Quantile Advantage Estimation for Entropy-Safe Reasoning [44.192277495613695]
  Reinforcement Learning with Verifiable Rewards (RLVR) strengthens LLM reasoning, but training often oscillates between entropy collapse and entropy explosion<n>We trace both hazards to the mean baseline used in value-free RL, which improperly penalizes negative-advantage samples under reward outliers.<n>We propose Quantile Advantage Estimation (QAE), replacing the mean with a group-wise K-quantile baseline.
  arXiv  Detail & Related papers  (2025-09-26T17:37:52Z)
• Causal Understanding by LLMs: The Role of Uncertainty [43.87879175532034]
  Recent papers show LLMs achieve near-random accuracy in causal relation classification.<n>We investigate whether pretraining exposure to causal examples improves causal understanding.
  arXiv  Detail & Related papers  (2025-09-24T13:06:35Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think [51.0691253204425]
  We analyze intermediate reasoning steps, termed subthoughts, to answer two questions: Does the final answer reliably represent the model's optimal conclusion?<n>Our approach involves segmenting a reasoning trace into sequential subthoughts based on linguistic cues.<n>We find that aggregating these answers by selecting the most frequent one (the mode) often yields significantly higher accuracy compared to relying solely on the answer derived from the original complete trace.
  arXiv  Detail & Related papers  (2025-04-29T12:39:07Z)
• Corruption-Robust Offline Reinforcement Learning with General Function Approximation [60.91257031278004]
  We investigate the problem of corruption in offline reinforcement learning (RL) with general function approximation. Our goal is to find a policy that is robust to such corruption and minimizes the suboptimality gap with respect to the optimal policy for the uncorrupted Markov decision processes (MDPs)
  arXiv  Detail & Related papers  (2023-10-23T04:07:26Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)
• Linear Contextual Bandits with Adversarial Corruptions [91.38793800392108]
  We study the linear contextual bandit problem in the presence of adversarial corruption. We present a variance-aware algorithm that is adaptive to the level of adversarial contamination $C$.
  arXiv  Detail & Related papers  (2021-10-25T02:53:24Z)
• On $\ell_p$-norm Robustness of Ensemble Stumps and Trees [83.81523991945018]
  We develop an efficient programming based algorithm for sound verification of ensemble stumps. We demonstrate the first certified defense method for training ensemble stumps and trees with respect to $ell_p$ norm perturbations.
  arXiv  Detail & Related papers  (2020-08-20T03:42:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.