Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching

• URL: http://arxiv.org/abs/2608.22332v1
• Date: Sun, 23 Aug 2026 10:05:16 GMT
• Title: Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching
• Abstract summary: We investigate where Chain-of-Thought (CoT)-related causal effects emerge across the generated reasoning trajectory.<n>We introduce a sequential activation patching framework that traces CoT-conditioned attention-head activations across token positions.<n>Our results provide evidence for distributed reasoning-support sub-circuits associated with CoT-conditioned exemplars.
• Score: 0.9940728137241215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) demonstrate remarkable problem-solving capabilities when guided by Chain-of-Thought (CoT) prompting, yet the internal mechanisms underlying these improvements remain poorly understood. In this work, we investigate where CoT-related causal effects emerge across the generated reasoning trajectory and which attention heads carry signals that contribute to final-answer computation. Because CoT reasoning unfolds over multiple generated tokens, standard activation patching at a single static token position is insufficient to characterize these temporally distributed effects. To address this limitation, we introduce a sequential activation patching framework that traces CoT-conditioned attention-head activations across token positions and aggregates their effects using Part-of-Speech-guided analysis. We further introduce Sequential Multi-Head Patching to evaluate the joint contribution of distributed head sets, together with cross-question and random activation controls. Targeted zero-ablation experiments show that the identified heads are functionally important for successful answer generation and affect several overlapping mechanisms, including reasoning-trajectory maintenance, answer anchoring, exemplar-target separation, and numerical generation. Overall, our results provide evidence for distributed reasoning-support sub-circuits associated with CoT-conditioned computation.

Related papers

• Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to Intervention [33.40772052822545]
  This paper bridges the gap between mechanistic interpretability and actionable control.<n>We first present a systematic analysis using structural, causal, and geometric probes.<n>We then operationalize these interpretability insights into a suite of training-free, decode-time interventions.
  arXiv  Detail & Related papers  (2026-05-31T13:52:24Z)
• The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity [22.040532283910522]
  This work provides a textitmechanistic explanation for the phenomenon.<n>We trace its root to the value aggregation process inherent in self-attention.<n>As a proof of concept, we propose textithead-wise RMSNorm, an architectural modification that stabilizes value aggregation outputs during pre-training.
  arXiv  Detail & Related papers  (2026-05-07T17:28:55Z)
• Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models [0.0]
  Chain-of-thought (CoT) prompting is widely used as a reasoning aid and is often treated as a transparency mechanism.<n>We introduce a causal, layerwise audit of CoT faithfulness based on activation patching.<n>We find that CoT-specific influence is typically depth-localized into narrow "reasoning windows"
  arXiv  Detail & Related papers  (2026-02-03T20:27:49Z)
• Seeing Through the Chain: Mitigate Hallucination in Multimodal Reasoning Models via CoT Compression and Contrastive Preference Optimization [78.94590726578014]
  multimodal reasoning models (MLRMs) remain prone to hallucinations, and effective solutions are still underexplored.<n>We propose C3PO, a training-based mitigation framework comprising textbfCompression and textbfPreference textbfOptimization.
  arXiv  Detail & Related papers  (2026-02-03T11:00:55Z)
• Accelerating Structured Chain-of-Thought in Autonomous Vehicles [55.78214218065611]
  Chain-of-Thought (CoT) reasoning enhances the decision-making capabilities of vision-language-action models in autonomous driving.<n>FastDriveCoT is a novel parallel decoding method that accelerates template-structured CoT.
  arXiv  Detail & Related papers  (2026-02-02T22:14:26Z)
• On the Paradoxical Interference between Instruction-Following and Task Solving [50.75960598434753]
  Instruction following aims to align Large Language Models (LLMs) with human intent by specifying explicit constraints on how tasks should be performed.<n>We reveal a counterintuitive phenomenon: instruction following can paradoxically interfere with LLMs' task-solving capability.<n>We propose a metric, SUSTAINSCORE, to quantify the interference of instruction following with task solving.
  arXiv  Detail & Related papers  (2026-01-29T17:48:56Z)
• Decoupling the Effect of Chain-of-Thought Reasoning: A Human Label Variation Perspective [60.45433515408158]
  We show that long Chain-of-Thought (CoT) serves as a decisive decision-maker for the top option but fails to function as a granular distribution calibrator for ambiguous tasks.<n>We observe a distinct "decoupled mechanism": while CoT improves distributional alignment, final accuracy is dictated by CoT content.
  arXiv  Detail & Related papers  (2026-01-06T16:26:40Z)
• Efficient Thought Space Exploration through Strategic Intervention [54.35208611253168]
  We propose a novel Hint-Practice Reasoning (HPR) framework that operationalizes this insight through two synergistic components.<n>The framework's core innovation lies in Distributional Inconsistency Reduction (DIR), which dynamically identifies intervention points.<n> Experiments across arithmetic and commonsense reasoning benchmarks demonstrate HPR's state-of-the-art efficiency-accuracy tradeoffs.
  arXiv  Detail & Related papers  (2025-11-13T07:26:01Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)
• Don't Take Things Out of Context: Attention Intervention for Enhancing Chain-of-Thought Reasoning in Large Language Models [32.71672086718058]
  Few-shot Chain-of-Thought (CoT) significantly enhances the reasoning capabilities of large language models (LLMs)<n>We observe that isolated segments, words, or tokens within CoT demonstrations can unexpectedly disrupt the generation process of LLMs.<n>We propose a Few-shot Attention Intervention method (FAI) that dynamically analyzes the attention patterns of demonstrations to accurately identify these tokens.
  arXiv  Detail & Related papers  (2025-03-14T07:46:33Z)
• Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs [63.36637269634553]
  We introduce a novel approach where LLMs are fine-tuned to generate a sequence of Diverse Chains of Thought (DCoT) within a single inference step.<n>We show that fine-tuning on DCoT improves performance over the CoT baseline across model families and scales.<n>Our work is also significant because both quantitative analyses and manual evaluations reveal the observed gains stem from the models' ability to refine an initial reasoning chain.
  arXiv  Detail & Related papers  (2024-07-03T15:01:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.