How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories

• URL: http://arxiv.org/abs/2607.28674v1
• Date: Tue, 28 Jul 2026 06:15:51 GMT
• Title: How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories
• Abstract summary: We propose Step-Aware Reasoning Energy (SARE), a geometric framework that quantifies effort at the granularity of individual CoT steps.<n>We find that reasoning energy is highly non-uniform across step types, exhibiting phase-like transitions invisible to trajectory-level metrics.<n>SARE-based features match or outperform output-based confidence baselines in most settings.
• Score: 51.16651846289446
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how computational effort is allocated across individual chain-of-thought (CoT) reasoning steps remains an open challenge: existing interpretability methods rely on output-level signals or collapse processing depth into a single trajectory-level scalar, leaving step-wise effort opaque. We propose Step-Aware Reasoning Energy (SARE), a geometric framework that quantifies effort at the granularity of individual CoT steps via Centered Kernel Alignment (CKA) between Gram matrices of token hidden states across adjacent transformer layers, capturing inter-token relational structure without requiring eigenvector alignment or cluster correspondence. SARE further contextualizes this energy within reasoning's semantic progression by modeling CoT trajectories as transitions among latent semantic states. Across six reasoning benchmarks and three open-weight LLMs, we find that reasoning energy is highly non-uniform across step types, exhibiting phase-like transitions invisible to trajectory-level metrics; incorrect trajectories show systematically lower energy at critical reasoning junctions; and SARE-based features match or outperform output-based confidence baselines in most settings, indicating that internal geometric dynamics encode predictive information beyond surface-level signals.

Related papers

• From Self-Attention to Connection Laplacian: A Unified Operator View of Transformers [53.505136935517925]
  Self-attention is a ubiquitous primitive in modern sequence models, yet its operator-level geometry is only partially understood.<n>We prove that single-head attention (SHA) is exactly a connection propagation step with constant transport.<n>We find that trained Transformers across scales (from 124M to 8B) and structures (encoder/decoder) exhibit geometric structure consistent with our theory.
  arXiv  Detail & Related papers  (2026-07-12T09:44:54Z)
• Interpreting Latent CoT Reasoning as Dynamical Systems [2.2448294058653455]
  We model latent token sequences as trajectories in representation space.<n>We show that latent CoT exhibits structured, non-random dynamics with two distinct stability classes.<n>This framework advances the interpretability of latent CoT reasoning dynamics.
  arXiv  Detail & Related papers  (2026-06-20T16:02:55Z)
• EIVE: End-to-End Instance-Specific Visual Explanations for Detection Transformers [7.91708974258006]
  We propose an End-to-end Instance-specific Visual Explanation framework (EIVE)<n>EIVE directly generates instance-level saliency maps following the forward pass of Detection Transformer (DETR)-like models.<n> Experiments on MS 2017, ExDark, and Cityscapes demonstrate that EIVE produces high-quality instance-level saliency maps.
  arXiv  Detail & Related papers  (2026-06-01T02:56:31Z)
• Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs [84.3271821505699]
  Chain of Modality (CoM) is an agentic framework that transitions multimodal fusion from passive concatenation to dynamic orchestration.<n> operating in either a training-free or a data-efficient SFT setting, CoM achieves robust and consistent generalization across diverse benchmarks.
  arXiv  Detail & Related papers  (2026-04-16T01:21:14Z)
• Thinking Deeper, Not Longer: Depth-Recurrent Transformers for Compositional Generalization [1.5736899098702974]
  We propose a depth-recurrent Transformer that decouples computational depth from parameter count.<n>Our architecture incorporates three mechanisms to make deep recurrence (20+ steps) stable.<n>We observe a clear emphcomputational frontier, where performance transitions from chance to near-perfect as thinking steps scale with task complexity.
  arXiv  Detail & Related papers  (2026-03-23T08:06:45Z)
• No Global Plan in Chain-of-Thought: Uncover the Latent Planning Horizon of LLMs [65.783709850324]
  This work stems from prior complementary observations on the dynamics of Chain-of-Thought (CoT): Large Language Models (LLMs)<n>LLMs are shown latent planning of subsequent reasoning prior to CoT emergence, thereby diminishing the significance of explicit CoT.<n>We investigate the latent planning strength of LLMs, through our probing method, Tele-Lens, applying to hidden states across diverse task domains.
  arXiv  Detail & Related papers  (2026-02-02T13:46:56Z)
• Beyond Dense States: Elevating Sparse Transcoders to Active Operators for Latent Reasoning [25.054823575096332]
  We propose LSTR (Latent Sparse Transcoder Reasoning), a latent reasoning framework that elevates sparse transcoders into active reasoning operators.<n>At its core, LSTR employs a Latent Transition Transcoder (LTT) with a residual skip architecture that decouples linear manifold transport from sparse semantic updates.<n>Extensive experiments show that LSTR preserves reasoning accuracy and compression efficiency while substantially improving interpretability.
  arXiv  Detail & Related papers  (2026-02-02T06:08:35Z)
• Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics [69.00587226225232]
  We introduce a state-aware transition framework that abstracts CoT trajectories into structured latent dynamics.<n>To characterize the global structure of reasoning, we model their progression as a Markov chain.<n>This abstraction supports a range of analyses, including semantic role identification, temporal pattern visualization, and consistency evaluation.
  arXiv  Detail & Related papers  (2025-08-29T18:53:31Z)
• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition [57.51370433303236]
  Chain-of-thought (CoT) reasoning enables large language models to break down complex problems into interpretable intermediate steps.<n>We introduce groundingS, a framework that formulates CoT reasoning as a Markov decision process (MDP) with latent state transitions.<n>We show improvements in reasoning accuracy, diversity, and exploration efficiency across benchmark reasoning tasks.
  arXiv  Detail & Related papers  (2025-07-10T21:32:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.