Daily Paper Briefing

Abstract Overview

This paper investigates the degree to which privileged information (PI) contributes to on-policy self-distillation (OPSD) beyond the baseline gains of cross-mode distillation itself. To isolate these effects, the authors develop AMPLE-Math, a suite of 5,319 mathematical problems featuring six answer-matched reasoning views ranging from Answer Only to Full Trace. Using a frozen, thinking-enabled teacher to supervise student rollouts, they evaluate student performance across checkpoints in Qwen3-1.7B and SmolLM3-3B. The findings reveal that substantial gains occur even in reference-free distillation, with privileged references offering varying and model-dependent additional benefits. Furthermore, teacher-student dynamics demonstrate that the supervision's effectiveness depends heavily on student response trajectories and training duration rather than token-level loss tweaks alone.

Novelty

The study introduces a rigorous experimental framework to disentangle baseline cross-mode distillation benefits from the incremental value of privileged teacher references in OPSD. It also contributes AMPLE-Math, a dataset providing six audited, answer-matched reasoning representations per problem to evaluate reasoning density while holding ground-truth answers constant.

Results

In Qwen3-1.7B, reference-free cross-mode distillation captures most of the performance gain under thinking-enabled evaluation, with Clean Solution offering a modest additional advantage of 1.30 points at step 100. In SmolLM3-3B, Full Trace adds 2.0 points over the reference-free baseline at step 50, though performance across configurations deteriorates below the base model by step 100. Across both model families, replacing direct-response training rollouts with thinking-enabled rollouts reverses gains into performance losses under an identical evaluation setup.

Key Points

1. AMPLE-Math provides 5,319 math problems with six answer-matched reasoning views to isolate the impact of reference reasoning format on teacher supervision.
2. A major portion of OPSD performance gains stems from cross-mode distillation between direct-response rollouts and thinking-enabled teacher scoring rather than access to full solution traces.
3. Switching student rollouts from direct-response to thinking-enabled trajectories reverses distillation gains, and token-level loss interventions show that apparent loss-coverage benefits are largely driven by early stopping.

References

• arXiv: https://arxiv.org/abs/2609.20612v1
• Fugu-MT: https://fugumt.com/fugumt/paper_check/2609.20612v1
• Hugging Face Papers: https://huggingface.co/papers/2609.20612
• GitHub: https://github.com/xiuyuz/opsd-reference-study
• Hugging Face: https://huggingface.co/datasets/xiuyuz/ample-math