Reasoning Traces Shape Outputs but Models Won't Say So

• URL: http://arxiv.org/abs/2603.20620v1
• Date: Sat, 21 Mar 2026 03:31:36 GMT
• Title: Reasoning Traces Shape Outputs but Models Won't Say So
• Abstract summary: We introduce Thought Injection, a method that injects synthetic reasoning snippets into a model's think> trace.<n>We find that injected hints reliably alter outputs, confirming that reasoning traces causally shape model behavior.<n>When asked to explain their changed answers, models overwhelmingly refuse to disclose the influence.
• Score: 11.928107300132062
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Can we trust the reasoning traces that large reasoning models (LRMs) produce? We investigate whether these traces faithfully reflect what drives model outputs, and whether models will honestly report their influence. We introduce Thought Injection, a method that injects synthetic reasoning snippets into a model's <think> trace, then measures whether the model follows the injected reasoning and acknowledges doing so. Across 45,000 samples from three LRMs, we find that injected hints reliably alter outputs, confirming that reasoning traces causally shape model behavior. However, when asked to explain their changed answers, models overwhelmingly refuse to disclose the influence: overall non-disclosure exceeds 90% for extreme hints across 30,000 follow-up samples. Instead of acknowledging the injected reasoning, models fabricate aligned-appearing but unrelated explanations. Activation analysis reveals that sycophancy- and deception-related directions are strongly activated during these fabrications, suggesting systematic patterns rather than incidental failures. Our findings reveal a gap between the reasoning LRMs follow and the reasoning they report, raising concern that aligned-appearing explanations may not be equivalent to genuine alignment.

Related papers

• Evading Chain-of-Thought Monitoring Through Model Poisoning [1.6951239729573633]
  Chain-of-thought (CoT) monitoring is an increasingly important component of AI safety stacks.<n>This work studies the limits of CoT monitoring through the lens of model poisoning.<n> backdoors can be implanted into reasoning models to elicit an attacker-chosen behavior while their CoT traces appear entirely benign.
  arXiv  Detail & Related papers  (2026-08-03T19:32:13Z)
• Pando: Do Interpretability Methods Work When Models Won't Explain Themselves? [53.07826484214082]
  We introduce Pando, a model-organism benchmark that breaks this confound via an explanation axis.<n>Pando predicts held-out model decisions from $10$ labeled query-response pairs.<n>When explanations are faithful, black-box elicitation matches or exceeds all white-box methods.
  arXiv  Detail & Related papers  (2026-04-13T06:42:24Z)
• How to Steal Reasoning Without Reasoning Traces [11.96740610715437]
  Many large language models (LLMs) use reasoning to generate responses but do not reveal their full reasoning traces.<n>We introduce trace inversion models that, given only the inputs, answers, and (optionally) reasoning summaries exposed by a target model, generate detailed, synthetic reasoning traces.
  arXiv  Detail & Related papers  (2026-03-07T15:50:44Z)
• To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks [56.11584171938381]
  Theory of Mind (ToM) assesses whether models can infer hidden mental states such as beliefs, desires, and intentions.<n>Recent progress in Large Reasoning Models (LRMs) has boosted step-by-step inference in mathematics and coding.<n>We present a systematic study of nine advanced Large Language Models (LLMs) comparing reasoning models with non-reasoning models.
  arXiv  Detail & Related papers  (2026-02-11T08:16:13Z)
• Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models [72.4149653187766]
  We propose a Reasoner-Verifier framework named Adrialversa Reasoning RAG (ARR)<n>The Reasoner and Verifier engage in reasoning on retrieved evidence and critiquing each other's logic while being guided by process-aware advantage.<n> Experiments on multiple benchmarks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2026-01-08T06:57:03Z)
• Analyzing Reasoning Consistency in Large Multimodal Models under Cross-Modal Conflicts [74.47786985522762]
  We identify a critical failure mode termed textual inertia, where models tend to blindly adhere to the erroneous text while neglecting conflicting visual evidence.<n>We propose the LogicGraph Perturbation Protocol that structurally injects perturbations into the reasoning chains of diverse LMMs.<n>Results reveal that models successfully self-correct in less than 10% of cases and predominantly succumb to blind textual error propagation.
  arXiv  Detail & Related papers  (2026-01-07T16:39:34Z)
• Distortion Instead of Hallucination: The Effect of Reasoning Under Strict Constraints [0.0]
  Reasoning capabilities have received attention as a self-verification process to improve output reliability.<n>We conduct experiments under strict constraints to examine the effect of reasoning across multiple models.<n>Our results reveal a problematic trade-off between constraint compliance and factual accuracy.
  arXiv  Detail & Related papers  (2026-01-04T11:35:39Z)
• Thought Branches: Interpreting LLM Reasoning Requires Resampling [10.978878422025232]
  We argue that studying a single sample is inadequate for understanding causal influence and the underlying computation.<n>We present case studies using resampling to investigate model decisions.
  arXiv  Detail & Related papers  (2025-10-31T14:02:37Z)
• The Ends Justify the Thoughts: RL-Induced Motivated Reasoning in LLMs [2.583082967853897]
  We find that motivated reasoning can be detected by most frontier reasoning models.<n>We find that as models become more sophisticated, their motivated reasoning may become increasingly difficult for monitors to detect.
  arXiv  Detail & Related papers  (2025-10-20T00:24:08Z)
• From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models [48.01707022738742]
  We conduct a three-stage investigation into the interplay between reasoning and answer generation in three distilled DeepSeek R1 models.<n>We demonstrate that including explicit reasoning consistently improves answer quality across diverse domains.<n>Our results show that perturbations to key reasoning tokens can reliably alter the final answers.
  arXiv  Detail & Related papers  (2025-09-28T06:32:21Z)
• Socratic-MCTS: Test-Time Visual Reasoning by Asking the Right Questions [100.41062461003389]
  We show that framing reasoning as a search process helps the model "connect the dots" between fragmented knowledge and produce extended reasoning traces in non-reasoning models.<n>We evaluate our method across three benchmarks and observe consistent improvements.
  arXiv  Detail & Related papers  (2025-06-10T15:51:16Z)
• The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning Models [38.11937119873932]
  Reasoning models often exhibit overthinking, characterized by redundant reasoning steps.<n>We identify internal bias elicited by the input question as a key trigger of such behavior.
  arXiv  Detail & Related papers  (2025-05-22T09:35:52Z)
• Rethinking harmless refusals when fine-tuning foundation models [0.8571111167616167]
  We investigate the degree to which fine-tuning in Large Language Models (LLMs) effectively mitigates versus merely conceals undesirable behavior. We identify a pervasive phenomenon we term emphreason-based deception, where models either stop producing reasoning traces or produce seemingly ethical reasoning traces that belie the unethical nature of their final outputs.
  arXiv  Detail & Related papers  (2024-06-27T22:08:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.