Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs

• URL: http://arxiv.org/abs/2608.05660v1
• Date: Thu, 06 Aug 2026 06:58:10 GMT
• Title: Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs
• Abstract summary: Language models are increasingly used for tasks that require verifiable reasoning.<n>Recent trajectory-based methods seek this signal in layerwise residual-stream displacements.<n>We propose a three-stream detector that combines motion with two restricted views of location.
• Score: 6.839620870759728
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As language models are increasingly used for tasks that require verifiable reasoning, reliably distinguishing sound reasoning from flawed reasoning has become an important practical problem. Recent trajectory-based methods seek this signal in layerwise residual-stream displacements, which capture how representations change while attenuating some stable, token-specific information. However, displacement omits the state from which an update originates, whereas restoring the full state risks reintroducing shortcut-prone information. We identify this trade-off and propose a three-stream detector that combines motion with two restricted views of location. A coarse region reader based on vector quantization and a fine direction reader over normalized multi-layer states. This design restores enough state context to interpret the motion without returning to full-state probing. On reasoning benchmarks unseen during training, our method improves selection accuracy by up to 12% over the displacement-only state of the art and 21% over single-layer probing baselines. Although trained only on reasoning benchmarks, it also reads factual completion and fact verification, ahead of every detector we compare against, which places the signal on correctness rather than on a kind of reasoning. Ablations further show that motion, region, and direction provide complementary signals. These results suggest that reasoning validity is better read from state-conditioned motion than from either static states or decontextualized trajectories alone.

Related papers

• Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth [50.62320148925983]
  Vision--language models (VLMs) can answer chart questions accurately, but output accuracy does not show how they combine the evidence needed to recover an exact value.<n>We study vertical-bar value reading with controlled counterfactual activation patching in Qwen2.5VL-7B-Instruct and InternVL3.5-8B.
  arXiv  Detail & Related papers  (2026-09-12T06:28:11Z)
• PERSIST: Persistent-State Discrimination for Shot Boundary Detection [2.2822910284300355]
  Shot boundary detection (SBD) is widely treated as the localisation of local visual discontinuities.<n>We reformulate SBD as boundary semantic discrimination.<n>PerSIST turns every decision into an inspectable one.
  arXiv  Detail & Related papers  (2026-08-29T14:23:00Z)
• Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering [44.273459587532535]
  We analyze and discover latent transition dynamics in the reasoning traces from Large Language Models.<n>We show that failure trajectories get stuck in self-loops, exhausting the token budget without progress toward the final answer.<n>We propose SOPHIA: Steering Of reasoning Processes via Hidden-state Intervention and Activations.
  arXiv  Detail & Related papers  (2026-07-20T16:01:57Z)
• Where Does Reasoning Break? Step-Level Hallucination Detection via Hidden-State Transport Geometry [3.2848713528308817]
  Large language models hallucinate during multi-step reasoning, but most existing detectors operate at the trace level.<n>We operationalize this view with a label-conditioned teacher that builds a trace-specific contrastive PCA lens.<n>We prove that contrastive PCA is the optimal projection for a transport-separation objective between first error and correct states.
  arXiv  Detail & Related papers  (2026-05-13T16:48:48Z)
• Rethinking State Tracking in Recurrent Models Through Error Control Dynamics [26.942965240880515]
  We show that affine recurrent networks cannot correct errors along state-separating subspaces once they preserve state representations.<n>We prove that robust state tracking is determined not only by an architecture's theoretical expressivity but crucially by its error control.
  arXiv  Detail & Related papers  (2026-05-08T13:59:15Z)
• Correct Answers from Sound Reasoning: Verifiable Process Supervision for Language Models [94.68358825189738]
  We propose a post-training framework for verifiable domains that jointly optimize prediction accuracy and reasoning quality.<n>We evaluate VPS on chess, a controlled testbed where reasoning steps can be deterministically verified against engine signals.<n>VPS preserves accuracy while significantly improving reasoning quality, reducing win-rate error by up to 30% and restoring consistency to near saturation.
  arXiv  Detail & Related papers  (2026-04-03T15:19:46Z)
• SEAL: Steerable Reasoning Calibration of Large Language Models for Free [58.931194824519935]
  Large Language Models (LLMs) have demonstrated compelling capabilities for complex reasoning tasks via the extended chain-of-thought (CoT) reasoning mechanism.<n>Recent studies reveal substantial redundancy in the CoT reasoning traces, which negatively impacts model performance.<n>We introduce SEAL, a training-free approach that seamlessly calibrates the CoT process, improving accuracy while demonstrating significant efficiency gains.
  arXiv  Detail & Related papers  (2025-04-07T02:42:07Z)
• Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models [58.64449765678416]
  We introduce landscape of thoughts (LoT) to inspect the reasoning trajectories with certain reasoning methods on any multi-choice dataset.<n>LoT distinguishes between strong and weak models, correct and incorrect answers, as well as different reasoning tasks.<n>We showcase this advantage by adapting LoT to a lightweight verifier that evaluates the correctness of trajectories.
  arXiv  Detail & Related papers  (2025-03-28T06:09:51Z)
• Modeling State Shifting via Local-Global Distillation for Event-Frame Gaze Tracking [61.44701715285463]
  This paper tackles the problem of passive gaze estimation using both event and frame data. We reformulate gaze estimation as the quantification of the state shifting from the current state to several prior registered anchor states. To improve the generalization ability, instead of learning a large gaze estimation network directly, we align a group of local experts with a student network.
  arXiv  Detail & Related papers  (2024-03-31T03:30:37Z)
• A New Bandit Setting Balancing Information from State Evolution and Corrupted Context [52.67844649650687]
  We propose a new sequential decision-making setting combining key aspects of two established online learning problems with bandit feedback. The optimal action to play at any given moment is contingent on an underlying changing state which is not directly observable by the agent. We present an algorithm that uses a referee to dynamically combine the policies of a contextual bandit and a multi-armed bandit.
  arXiv  Detail & Related papers  (2020-11-16T14:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.