Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models

• URL: http://arxiv.org/abs/2603.26410v1
• Date: Fri, 27 Mar 2026 13:39:05 GMT
• Title: Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models
• Abstract summary: Extended-thinking models expose a second text-generation channel ("thinking tokens") alongside the user-visible answer.<n>This study examines 12 open-weight reasoning models on MMLU and GPQA questions paired with misleading hints.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extended-thinking models expose a second text-generation channel ("thinking tokens") alongside the user-visible answer. This study examines 12 open-weight reasoning models on MMLU and GPQA questions paired with misleading hints. Among the 10,506 cases where models actually followed the hint (choosing the hint's target over the ground truth), each case is classified by whether the model acknowledges the hint in its thinking tokens, its answer text, both, or neither. In 55.4% of these cases the model's thinking tokens contain hint-related keywords that the visible answer omits entirely, a pattern termed *thinking-answer divergence*. The reverse (answer-only acknowledgment) is near-zero (0.5%), confirming that the asymmetry is directional. Hint type shapes the pattern sharply: sycophancy is the most *transparent* hint, with 58.8% of sycophancy-influenced cases acknowledging the professor's authority in both channels, while consistency (72.2%) and unethical (62.7%) hints are dominated by thinking-only acknowledgment. Models also vary widely, from near-total divergence (Step-3.5-Flash: 94.7%) to relative transparency (Qwen3.5-27B: 19.6%). These results show that answer-text-only monitoring misses more than half of all hint-influenced reasoning and that thinking-token access, while necessary, still leaves 11.8% of cases with no verbalized acknowledgment in either channel.

Related papers

• Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning [55.264863369127774]
  Current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.<n>We propose Hidden-Align, an auxiliary loss function that aligns the last-layer hidden states of correct rollouts at the anchor token during RL training, with zero overhead in both training and inference.<n>On eight mathematical reasoning benchmarks, Hidden-Align improves average pass@1 over the DAPO baseline by 3.8, 6.2, and 5.4 percentage points on Qwen3-1.7B, 4B, and 14B respectively, with consistent pass@k gains across all three scales.
  arXiv  Detail & Related papers  (2026-06-02T06:51:15Z)
• Understanding and Mitigating Premature Confidence for Better LLM Reasoning [76.16007941549857]
  Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps.<n>We find such a signal in how the model's confidence evolves during reasoning.<n>We exploit this in progressive confidence shaping, a reinforcement learning objective that trains models to update their confidence as they reason rather than commit early.
  arXiv  Detail & Related papers  (2026-05-23T04:42:45Z)
• Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models? [0.0]
  Chain-of-thought (CoT) reasoning has been proposed as a transparency mechanism for large language models in safety-critical deployments.<n>This study tests 12 open-weight reasoning models spanning 9 architectural families on 498 multiple-choice questions.<n>Across 41,832 inference runs, overall faithfulness rates range from 39.7% (Seed-1.6-Flash) to 89.9% (DeepSeek-V3.2-Speciale) across model families.
  arXiv  Detail & Related papers  (2026-03-23T21:21:37Z)
• Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluation [0.0]
  Recent work on chain-of-thought faithfulness reports single aggregate numbers.<n>This paper provides evidence that faithfulness is not an objective, measurable property of a model.
  arXiv  Detail & Related papers  (2026-03-20T17:48:43Z)
• How to Steal Reasoning Without Reasoning Traces [11.96740610715437]
  Many large language models (LLMs) use reasoning to generate responses but do not reveal their full reasoning traces.<n>We introduce trace inversion models that, given only the inputs, answers, and (optionally) reasoning summaries exposed by a target model, generate detailed, synthetic reasoning traces.
  arXiv  Detail & Related papers  (2026-03-07T15:50:44Z)
• Oops, Wait: Token-Level Signals as a Lens into LLM Reasoning [61.76889440384448]
  discourse-like tokens such as "wait" and "therefore" in large language models (LLMs) have offered a unique window into their reasoning processes.<n>We analyze token-level signals through token probabilities across various models.
  arXiv  Detail & Related papers  (2026-01-24T11:43:09Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think [51.0691253204425]
  We analyze intermediate reasoning steps, termed subthoughts, to answer two questions: Does the final answer reliably represent the model's optimal conclusion?<n>Our approach involves segmenting a reasoning trace into sequential subthoughts based on linguistic cues.<n>We find that aggregating these answers by selecting the most frequent one (the mode) often yields significantly higher accuracy compared to relying solely on the answer derived from the original complete trace.
  arXiv  Detail & Related papers  (2025-04-29T12:39:07Z)
• Chain-of-Thought Reasoning In The Wild Is Not Always Faithful [3.048751803239144]
  Chain-of-Thought (CoT) reasoning has significantly advanced state-of-the-art AI capabilities.<n>We show that unfaithful CoT can also occur on realistic prompts with no artificial bias.
  arXiv  Detail & Related papers  (2025-03-11T17:56:30Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Are DeepSeek R1 And Other Reasoning Models More Faithful? [2.0429566123690455]
  We evaluate three reasoning models based on Qwen-2.5, Gemini-2, and DeepSeek-V3-Base.<n>We test whether models can describe how a cue in their prompt influences their answer to MMLU questions.<n> Reasoning models describe cues that influence them much more reliably than all the non-reasoning models tested.
  arXiv  Detail & Related papers  (2025-01-14T14:31:45Z)
• Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step [81.50681925980135]
  We propose a method to probe changes in the mind during the model's reasoning.<n>By analyzing patterns in mind change, we examine the correctness of the model's reasoning.<n>Our validation reveals that many responses, although correct in their final answer, contain errors in their reasoning process.
  arXiv  Detail & Related papers  (2024-06-23T15:50:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.