Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators

• URL: http://arxiv.org/abs/2608.29956v1
• Date: Sun, 30 Aug 2026 18:37:46 GMT
• Title: Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators
• Abstract summary: Hidden CoT Detection Score (HCDS) measures whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no-CoT.<n>HCDS does not directly observe or prove an unexposed reasoning trace.<n>Findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model.
• Score: 3.0130431031064613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models often answer complex reasoning questions without revealing intermediate steps, raising whether they reason latently or complete patterns. We propose the Hidden CoT Detection Score (HCDS), a comparative behavioral and mechanistic signal measuring whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no- CoT. Here, hidden CoT operationally denotes this neutral-prompt CoT-like alignment; HCDS does not directly observe or prove an unexposed reasoning trace. On GSM8K, HCDS is significantly positive for both Qwen3-4B variants (Thinking $+1.87$, $p = 1.2 \times 10^{-7}$; Instruct $+1.41$, $p = 1.9 \times 10^{-4}$), replicates across a different inference stack and quantization within $0.08$ ($+1.80$ and $+1.45$), and is not significantly positive in seven of eight length-adjusted calibration-control cells. The unadjusted score produces large positive scores on single-step arithmetic and numeric factual lookup. The variants also respond differently to no-CoT instructions: Instruct complies from the prompt alone, whereas Thinking continues reasoning and requires intervention. These findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model, consistent with but not proof of latent reasoning. HCDS thus investigates latent reasoning without relying on models' self-reported traces.

Related papers

• Selective Disclosure of Hidden Directives in Reasoning Models: Behavioral Asymmetry and Steering [0.9474649136535703]
  Chain-of-thought (CoT) reasoning traces are increasingly proposed as a mechanism for AI oversight.<n>This assumes CoT surfaces what a model is instructed to do regardless of instructions given.<n>We test this assumption along two axes.
  arXiv  Detail & Related papers  (2026-08-29T05:50:54Z)
• The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics [0.9668407688201359]
  Chain-of-thought (CoT) reasoning improves large language model (LLM) performance.<n>Existing approaches that leverage verbalized CoTs to monitor reasoning correctness largely evaluate the semantic correctness or consistency of individual intermediate steps.<n>We show that capability failures can manifest as distributed, task-dependent changes in the structure of visible reasoning.
  arXiv  Detail & Related papers  (2026-08-04T08:04:36Z)
• When Do LLM Agents Treat Surface Noise Differently from Semantic Noise? A 68-Cell Measurement Study with a Held-Out Trace-Level Validation [9.055086193088083]
  We document an empirical phenomenon in chain-of-thought and ReAct agents driven by ten large language models.<n>Mean-bearing perturbations alter final answers more often than presentation perturbations of comparable severity.
  arXiv  Detail & Related papers  (2026-05-25T15:57:11Z)
• Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models [0.0]
  Chain-of-thought (CoT) prompting is widely used as a reasoning aid and is often treated as a transparency mechanism.<n>We introduce a causal, layerwise audit of CoT faithfulness based on activation patching.<n>We find that CoT-specific influence is typically depth-localized into narrow "reasoning windows"
  arXiv  Detail & Related papers  (2026-02-03T20:27:49Z)
• From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs [58.02809208460186]
  We revisit this paradox using high-quality traces from DeepSeek-R1 as demonstrations.<n>We find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal.<n>We introduce Insight-to-solve (I2S), a sequential test-time procedure that turns demonstrations into explicit, reusable insights.
  arXiv  Detail & Related papers  (2025-09-27T08:59:31Z)
• SIM-CoT: Supervised Implicit Chain-of-Thought [108.30049193668083]
  Implicit Chain-of-Thought (CoT) methods offer a token-efficient alternative to explicit CoT reasoning in Large Language Models.<n>We identify a core latent instability issue when scaling the computational budget of implicit CoT.<n>We propose SIM-CoT, a plug-and-play training module that introduces step-level supervision to stabilize and enrich the latent reasoning space.
  arXiv  Detail & Related papers  (2025-09-24T17:01:32Z)
• How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding [3.8914132324834045]
  Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks.<n>But whether the generated "thoughts" reflect the true internal reasoning process is unresolved.<n>We present the first feature-level causal study of CoT faithfulness.
  arXiv  Detail & Related papers  (2025-07-24T10:25:46Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• Reasoning Models Don't Always Say What They Think [48.05987314492555]
  Chain-of-thought (CoT) allows monitoring a model's intentions and reasoning processes.<n>We evaluate CoT faithfulness of state-of-the-art reasoning models across 6 reasoning hints presented in prompts.
  arXiv  Detail & Related papers  (2025-05-08T16:51:43Z)
• A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning [48.51969964676017]
  Chain-of-Thought (CoT) holds a significant place in augmenting the reasoning performance for large language models. We propose a Read-and-Control approach for controlling the accuracy of CoT.
  arXiv  Detail & Related papers  (2024-06-18T04:07:13Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.