How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning

• URL: http://arxiv.org/abs/2604.19149v1
• Date: Tue, 21 Apr 2026 06:55:17 GMT
• Title: How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning
• Abstract summary: We analyze the answer-to-reasoning attention and observe a benign self-reading pattern aligned with correctness.<n>We propose a training-free steering method driven by Self-Reading Quality (SRQ) scores combining geometric metrics for process control with semantic metrics for content monitoring.
• Score: 14.803331273106906
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Thinking LLMs produce reasoning traces before answering. Prior activation steering work mainly targets on shaping these traces. It remains less understood how answer tokens actually read and integrate the reasoning to produce reliable outcomes. Focusing on quantitative reasoning, we analyze the answer-to-reasoning attention and observe a benign self-reading pattern aligned with correctness, characterized by a forward drift of the reading focus along the reasoning trace and a persistent concentration on key semantic anchors, whereas incorrect solutions exhibit diffuse and irregular attention pattern. We interpret this as internal certainty during answer decoding, where the model commits to a viable solution branch and integrates key evidence. Following this, we propose a training-free steering method driven by Self-Reading Quality (SRQ) scores combining geometric metrics for process control with semantic metrics for content monitoring. SRQ selects data to build steering vectors that guide inference toward benign self-reading and away from uncertain and disorganized reading. Experiments show that our method yields consistent accuracy gains.

Related papers

• How Does Reasoning Flow? Tracing Attention-Induced Information Flow for Targeted RL in LLMs [52.712343092801326]
  FlowTracer is an RL framework that traces answer-targeted reasoning flow on an attention-weighted directed acyclic graph.<n>It scores tokens by flow throughput, revealing high-impact hubs and aggregation checkpoints that mediate long-range dependencies.<n>These derived importances are used to shape token-level rewards, enabling learning signals to focus precisely on the tokens that route information toward correct answers.
  arXiv  Detail & Related papers  (2026-06-09T09:56:51Z)
• AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency [40.19713302778418]
  We introduce AtManRL, a method that leverages differentiable attention manipulation to learn more faithful reasoning through reinforcement learning.<n>We integrate this saliency reward with outcome-based rewards within the GRPO framework to jointly optimize for correctness and interpretability.
  arXiv  Detail & Related papers  (2026-04-17T15:27:35Z)
• FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoning [49.65751420291115]
  Chain-of-Thought (CoT) prompting has improved LLM reasoning, but models often generate explanations that appear coherent while containing unfaithful intermediate steps.<n>We propose FACT-E, a causality-inspired framework for evaluating CoT quality.<n>We show that FACT-E improves reasoning-trajectory selection and yields stronger in-context learnings.
  arXiv  Detail & Related papers  (2026-04-12T15:35:08Z)
• Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution [79.98699884805636]
  Reasoning Execution by Multiple Listeners (REMUL) is a multi-party reinforcement learning approach.<n>REMUL builds on the hypothesis that reasoning traces which other parties can follow will be more faithful.<n>Speakers are rewarded for producing reasoning that is clear to listeners.
  arXiv  Detail & Related papers  (2026-02-18T02:55:55Z)
• From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models [48.01707022738742]
  We conduct a three-stage investigation into the interplay between reasoning and answer generation in three distilled DeepSeek R1 models.<n>We demonstrate that including explicit reasoning consistently improves answer quality across diverse domains.<n>Our results show that perturbations to key reasoning tokens can reliably alter the final answers.
  arXiv  Detail & Related papers  (2025-09-28T06:32:21Z)
• SelfJudge: Faster Speculative Decoding via Self-Supervised Judge Verification [28.63435151584449]
  We propose SelfJudge, which trains judge verifiers via self-supervision of the target model.<n>Our method measures semantic preservation by assessing whether token-substituted responses preserve the meaning of original responses.
  arXiv  Detail & Related papers  (2025-09-26T02:21:12Z)
• Read Before You Think: Mitigating LLM Comprehension Failures with Step-by-Step Reading [2.073147245888634]
  Large Language Models (LLMs) often fail on complex reasoning tasks due to flawed question comprehension.<n>This paper presents a systematic investigation into these comprehension failures.<n>We introduce the Step-by-Step Reading (SSR) family of prompts.
  arXiv  Detail & Related papers  (2025-04-13T02:10:18Z)
• Zero-Shot Verification-guided Chain of Thoughts [64.862738244735]
  We focus on self-verification of self-generated reasoning steps via COT prompts in a completely zero-shot regime.<n>To explore this setting, we design a new zero-shot prompt, which we call COT STEP, to aid zero-shot decomposition of reasoning steps.<n>We evaluate the verifiers' ability to classify the correctness of reasoning chains and explore different ways to use verifier scores in guiding reasoning.
  arXiv  Detail & Related papers  (2025-01-21T03:52:54Z)
• Evaluating Human Alignment and Model Faithfulness of LLM Rationale [66.75309523854476]
  We study how well large language models (LLMs) explain their generations through rationales. We show that prompting-based methods are less "faithful" than attribution-based explanations.
  arXiv  Detail & Related papers  (2024-06-28T20:06:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.