Revisiting Complete Reasoning Traces for Post-Training

• URL: http://arxiv.org/abs/2609.07103v1
• Date: Mon, 07 Sep 2026 06:46:17 GMT
• Title: Revisiting Complete Reasoning Traces for Post-Training
• Abstract summary: Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability.<n>We show that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation.
• Score: 61.76889440384448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex, interwoven paths, which often include detours on the path toward the answer. However, it has been underexplored whether LLMs indeed benefit from learning complete trajectories in post-training, such as supervised fine-tuning (SFT). Starting from our pilot study, we find that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation. We analyze redundancy in reasoning trajectories through attention-based analyses and controlled token-removal studies, both of which show that intermediate tokens contribute minimally to final reasoning quality. This suggests that avoiding redundant information may allow LLMs to internally infer coherent alternatives by inferring missing steps from their internal knowledge, given known trajectory endpoints. Furthermore, we show that training LLMs using endpoints leads to consistent changes in reasoning behavior, and that it also benefits post-training methods based on reinforcement learning or on-policy distillation, highlighting the need to revisit complete reasoning traces. Code is available at https://github.com/naver-ai/revisiting-trace.

Related papers

• PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents [57.69188221227875]
  We propose a Privileged trAce Co-Training framework for multi-turn tool-use agents.<n>The key idea is to use expert traces only as training-time optimization signals rather than rollout-time hints.<n>We show that PACT consistently improves over strong SFT- and RL-based baselines.
  arXiv  Detail & Related papers  (2026-06-15T04:46:23Z)
• TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agents [62.969415789328025]
  LLM agents can pursue hidden malicious objectives through sequences of individually benign actions.<n>Existing approaches either evaluate complete trajectories in a single pass or partition them into independently scored windows.<n>We propose TRACE, a monitoring framework for long-horizon LLM agent trajectories.
  arXiv  Detail & Related papers  (2026-06-05T08:54:38Z)
• ThoughtFold: Folding Reasoning Chains via Introspective Preference Learning [69.64972562984882]
  We propose ThoughtFold, a framework that leverages fine-grained preference learning to redundant explorations for efficient reasoning.<n>We show that ThoughtFold significantly enhances efficiency.<n>It reduces the token usage of DeepSeek-R1-Distill-Qwen-7B by approximately 56% while maintaining state-of-the-art accuracy.
  arXiv  Detail & Related papers  (2026-06-02T11:21:27Z)
• SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models [51.99912169291891]
  Reinforcement learning (RL) has emerged as a promising paradigm for training reasoning-oriented models.<n>We propose a training paradigm that interleaves conventional RL with inverse reinforcement learning (IRL)<n>Our findings suggest that alternating between RL and IRL offers an effective pathway toward extending the exploration capacity of reasoning-oriented large language models.
  arXiv  Detail & Related papers  (2026-04-18T13:49:47Z)
• Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning [18.215893951726166]
  In environments with sparse or delayed rewards, reinforcement learning incurs high sample complexity.<n>This limitation has motivated the use of large language models (LLMs) for subgoal discovery and trajectory guidance.<n>We address these challenges by constructing a memory graph that encodes subgoals and trajectories from both LLM guidance and the agent's own successful rollouts.
  arXiv  Detail & Related papers  (2026-02-20T01:44:35Z)
• Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectory? [13.980638430366625]
  Reasoning LLMs are trained to verbalize their reasoning process, yielding strong gains on complex tasks.<n>A key prerequisite is the ability to assess the usefulness and build on another model's partial thinking.<n>This paper investigates the question: can standard solo-reasoning training pipelines deliver desired off-trajectory behaviors?
  arXiv  Detail & Related papers  (2025-10-07T19:42:50Z)
• Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning [87.7836502955847]
  We propose a novel self-rewarding reinforcement learning framework to enhance Large Language Model (LLM) reasoning.<n>Our key insight is that correct responses often exhibit consistent trajectory patterns in terms of model likelihood.<n>We introduce CoVo, an intrinsic reward mechanism that integrates Consistency and Volatility via a robust vector-space aggregation strategy.
  arXiv  Detail & Related papers  (2025-06-10T12:40:39Z)
• Temporal Sampling for Forgotten Reasoning in LLMs [14.895026967556088]
  We show that models often forget how to solve problems they previously answered correctly during training.<n>We introduce Temporal Sampling, a simple decoding strategy that draws outputs from multiple checkpoints along the training trajectory.<n>This approach recovers forgotten solutions without retraining or ensembling, and leads to substantial improvements in reasoning performance.
  arXiv  Detail & Related papers  (2025-05-26T16:39:52Z)
• PLMTrajRec: A Scalable and Generalizable Trajectory Recovery Method with Pre-trained Language Models [31.98187852227127]
  We propose a novel trajectory recovery model called PLMTrajRec.<n>It leverages the scalability of a pre-trained language model (PLM) and can be fine-tuned with only a limited set of dense trajectories.<n>To handle different sampling intervals in sparse trajectories, we first convert each trajectory's sampling interval and movement features into natural language representations.
  arXiv  Detail & Related papers  (2024-10-18T08:38:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.