Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs

• URL: http://arxiv.org/abs/2606.00642v1
• Date: Sat, 30 May 2026 09:37:55 GMT
• Title: Hidden Thoughts Are Not Secret: Reasoning Trace Exposure in LLMs
• Abstract summary: Reasoning traces have become a valuable form of learning signals for improving and transferring the capabilities of large language models.<n>The value of capability transfer has motivated many deployed systems with reasoning models to hide raw internal traces and expose at most summaries and answers to users.<n>We study whether such interface-level trace hiding prevents users from obtaining useful reasoning supervision through prompting.
• Score: 16.213231993962733
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reasoning traces have become a valuable form of learning signals for improving and transferring the capabilities of large language models. In particular, detailed traces can help distill reasoning behavior from stronger teacher models into weaker student models. The value of capability transfer has motivated many deployed systems with reasoning models to hide raw internal traces and expose at most summaries and answers to users. As a result, we ask whether such interface-level trace hiding prevents users from obtaining useful reasoning supervision through prompting. We study this question with Reasoning Exposure Prompting (REP), a lightweight in-context elicitation method that uses shadow-model-generated demonstrations wrapped in auxiliary code-like formats to raise user-visible reasoning traces from a victim model. Across the common reasoning dataset, different victim models, and different student model distillation, REP substantially increases similarity between exposed and REP-conditioned internal traces while preserving useful reasoning signals.

Related papers

• LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection [12.083572608650904]
  Reinforcement-learning (RL) post-training equips multimodal large reasoning models (MLRMs) with exploratory chains of thought (CoT)<n>We find that even when a sensitive fact is successfully unlearned from the final answer, the model may still reproduce it in its reasoning trace.<n>We propose LEMUR, a fully training-free, inference-time unlearning framework for natively RL-trained multimodal models.
  arXiv  Detail & Related papers  (2026-08-12T06:03:50Z)
• TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory [35.19442792860492]
  As reasoning trajectories grow, models may become less effective at using information established earlier in the context.<n>Existing approaches primarily address this problem by sustaining visual grounding throughout reasoning.<n>We introduce TRAM (TRajectory-derived Auxiliary Memory), a training-free method that augments standard decoding with an auxiliary memory pathway.
  arXiv  Detail & Related papers  (2026-08-03T08:57:00Z)
• Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification [55.357038267439684]
  Visual Re-Examination (VRE) is a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs.<n>VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings.
  arXiv  Detail & Related papers  (2026-03-27T12:22:13Z)
• Thinking with Deltas: Incentivizing Reinforcement Learning via Differential Visual Reasoning Policy [75.66913260900726]
  Reinforcement Learning with Verifiable Rewards has significantly advanced reasoning capabilities in Large Language Models.<n>Existing paradigms, driven by text-centric outcome rewards, encourage models to bypass visual perception.<n>We propose textbfThinking with Deltas, a framework driven by a textbfDifferential Visual Reasoning Policy.
  arXiv  Detail & Related papers  (2026-01-11T08:25:34Z)
• Information-Preserving Reformulation of Reasoning Traces for Antidistillation [80.99448418790125]
  We propose an information-preserving antidistillation reformulation of reasoning traces.<n>A small auxiliary model is trained to perform this reformulation, incurring minimal computational overhead.<n>Experiments demonstrate that PART consistently disrupts distillation across student models of different sizes and types.
  arXiv  Detail & Related papers  (2025-10-13T15:42:11Z)
• Learning a Dense Reasoning Reward Model from Expert Demonstration via Inverse Reinforcement Learning [50.20267980386502]
  We learn a dense, token-level reward model for process supervision directly from expert demonstrations.<n>The learned reasoning reward serves two complementary roles: (i) it provides step-level feedback to optimise a reasoning policy during training; and (ii) it functions at inference as a critic to rerank sampled traces under fixed compute budgets.
  arXiv  Detail & Related papers  (2025-10-02T09:55:26Z)
• Neural Breadcrumbs: Membership Inference Attacks on LLMs Through Hidden State and Attention Pattern Analysis [9.529147118376464]
  Membership inference attacks (MIAs) reveal whether specific data was used to train machine learning models.<n>Our work explores how examining internal representations, rather than just their outputs, may provide additional insights into potential membership inference signals.<n>Our findings suggest that internal model behaviors can reveal aspects of training data exposure even when output-based signals appear protected.
  arXiv  Detail & Related papers  (2025-09-05T19:05:49Z)
• Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills [42.1825027925353]
  Large reasoning models (LRMs) have enabled strong chain-of-thought (CoT) generation through test-time computation.<n>We show that conventional unlearning algorithms, originally designed for non-reasoning models, are inadequate for LRMs.<n>We propose Reasoning-aware Representation Misdirection for Unlearning ($R2MU$), a novel method that effectively suppresses sensitive reasoning traces.
  arXiv  Detail & Related papers  (2025-06-15T20:54:23Z)
• Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models [27.142703756752997]
  We introduce MathIF, a benchmark for evaluating instruction-following in mathematical reasoning tasks.<n>Our empirical analysis reveals a consistent tension between scaling up reasoning capacity and maintaining controllability.<n>We show that even simple interventions can partially recover obedience, though at the cost of reasoning performance.
  arXiv  Detail & Related papers  (2025-05-20T18:18:01Z)
• Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrieval [51.53892300802014]
  We show that supervised neural information retrieval models are prone to learning sparse attention patterns over passage tokens. Using a novel targeted synthetic data generation method, we teach neural IR to attend more uniformly and robustly to all entities in a given passage.
  arXiv  Detail & Related papers  (2022-04-24T22:36:48Z)
• Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations [97.91630330328815]
  We conduct a crowdsourcing study, where participants interact with deception detection models that have been trained to distinguish between genuine and fake hotel reviews. We observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
  arXiv  Detail & Related papers  (2021-12-17T18:29:56Z)
• Unsupervised Noisy Tracklet Person Re-identification [100.85530419892333]
  We present a novel selective tracklet learning (STL) approach that can train discriminative person re-id models from unlabelled tracklet data. This avoids the tedious and costly process of exhaustively labelling person image/tracklet true matching pairs across camera views. Our method is particularly more robust against arbitrary noisy data of raw tracklets therefore scalable to learning discriminative models from unconstrained tracking data.
  arXiv  Detail & Related papers  (2021-01-16T07:31:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.