Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation

• URL: http://arxiv.org/abs/2606.00628v1
• Date: Sat, 30 May 2026 09:03:03 GMT
• Title: Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation
• Abstract summary: Self-distillation improves learning efficiency by rewriting reference answers as training data that better matches the model's own distribution.<n>But reference answers also introduce strong stylistic biases, causing the generative model to imitate surface forms rather than learn useful reasoning patterns.<n>We propose Distribution-Aligned Self-Distillation (DASD), which uses an answer-aware reference model to generate candidate tokens and dynamically filters them according to the base model's confidence.
• Score: 10.71966126081564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Self-distillation improves learning efficiency by rewriting reference answers as training data that better matches the model's own distribution. However, reference answers also introduce strong stylistic biases, causing the generative model to imitate surface forms rather than learn useful reasoning patterns. We observe that the rewriting data contains a large number of high-perplexity (PPL) tokens, coming from two distinct sources: beneficial knowledge-enhancing logical corrections, and harmful stylistic drift induced by reference imitation. Treating all such tokens equally can disrupt the base model's original distribution and degrade performance, especially on difficult reasoning tasks. To address this, we propose Distribution-Aligned Self-Distillation (DASD), which uses an answer-aware reference model to generate candidate tokens and dynamically filters them according to the base model's confidence. DASD preserves tokens that encode useful logical knowledge while suppressing distributionally misaligned style noise. Experiments on math, code, and commonsense reasoning benchmarks show that DASD consistently outperforms competitive baselines, reduces high-PPL tokens, and improves robustness across tasks of varying difficulty.

Related papers

• AMR-SD: Asymmetric Meta-Reflective Self-Distillation for Token-Level Credit Assignment [39.63424981516754]
  Asymmetric Meta-Reflective Self-Distillation (AMR-SD)<n>We introduce Causal Information Gain (CIG) with an asymmetric, ReLU-gated threshold to translate these reflections into sparse, highly precise token-level advantage modulations.<n> Experiments across scientific, mathematical, and tool-use benchmarks demonstrate that AMR-SD significantly outperforms existing baselines.
  arXiv  Detail & Related papers  (2026-05-18T15:14:34Z)
• MixSD: Mixed Contextual Self-Distillation for Knowledge Injection [29.7616760417696]
  Supervised fine-tuning (SFT) is widely used to inject new knowledge into language models.<n>We argue this forgetting arises because fine-tuning targets from humans or external systems diverge from the model's autoregressive distribution.<n>We propose MixSD, a simple external-teacher-free method for distribution-aligned knowledge injection.
  arXiv  Detail & Related papers  (2026-05-16T07:57:09Z)
• Not all tokens contribute equally to diffusion learning [16.928860227988086]
  Conditional diffusion models often neglect semantically important tokens during inference, leading to biased or incomplete generations.<n>We propose Distribution-Aware Rectification and Ensemble Spatial Alignment (DARE), a unified framework that improves semantic guidance from perspectives of distributional debiasing and consistency.<n>DARE consistently improves semantic alignment, achieving significant gains over existing approaches.
  arXiv  Detail & Related papers  (2026-04-08T12:45:21Z)
• CORE: Context-Robust Remasking for Diffusion Language Models [51.59514489363897]
  We propose Context-Robust Remasking (CORE), a training-free framework for inference-time revision.<n>Rather than trusting static token probabilities, CORE identifies context-brittle tokens by probing their sensitivity to targeted masked-context perturbations.<n>On LLaDA-8B-Base, CORE delivers consistent improvements across reasoning and code benchmarks, outperforming compute-matched baselines and improving MBPP by up to 9.2 percentage points.
  arXiv  Detail & Related papers  (2026-02-04T00:12:30Z)
• Did Models Sufficient Learn? Attribution-Guided Training via Subset-Selected Counterfactual Augmentation [61.248535801314375]
  Subset-Selected Counterfactual Augmentation (SS-CA)<n>We develop Counterfactual LIMA to identify minimal spatial region sets whose removal can selectively alter model predictions.<n>Experiments show that SS-CA improves generalization on in-distribution (ID) test data and achieves superior performance on out-of-distribution (OOD) benchmarks.
  arXiv  Detail & Related papers  (2025-11-15T08:39:22Z)
• Mitigating Spurious Correlations Between Question and Answer via Chain-of-Thought Correctness Perception Distillation [25.195244084313114]
  Chain-of-Thought Correctness Perception Distillation (CoPeD) aims to improve the reasoning quality of the student model.<n>CoPeD encourages the student model to predict answers based on correct rationales and revise them when they are incorrect.
  arXiv  Detail & Related papers  (2025-09-06T05:33:17Z)
• Reinforced Context Order Recovery for Adaptive Reasoning and Planning [23.229513376337607]
  Current causal and diffusion models encounter difficulties in problems that require adaptive token generation orders to solve tractably.<n>Motivated by this, we propose Reinforced Context Order Recovery (ReCOR), a reinforcement-learning-based framework to extract adaptive, data-dependent token generation orders.
  arXiv  Detail & Related papers  (2025-08-18T16:42:55Z)
• Continuous Visual Autoregressive Generation via Score Maximization [69.67438563485887]
  We introduce a Continuous VAR framework that enables direct visual autoregressive generation without vector quantization.<n>Within this framework, all we need is to select a strictly proper score and set it as the training objective to optimize.
  arXiv  Detail & Related papers  (2025-05-12T17:58:14Z)
• Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability [53.51560766150442]
  Critical tokens are elements within reasoning trajectories that significantly influence incorrect outcomes.<n>We present a novel framework for identifying these tokens through rollout sampling.<n>We show that identifying and replacing critical tokens significantly improves model accuracy.
  arXiv  Detail & Related papers  (2024-11-29T18:58:22Z)
• DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalization [44.291382840373]
  This paper addresses the challenge of out-of-distribution generalization in graph machine learning. Traditional graph learning algorithms falter in real-world scenarios where this assumption fails. A principal factor contributing to this suboptimal performance is the inherent simplicity bias of neural networks.
  arXiv  Detail & Related papers  (2024-08-08T12:08:55Z)
• Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation [63.180725016463974]
  Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice. We introduce a novel noisy correspondence learning framework, namely textbfSelf-textbfReinforcing textbfErrors textbfMitigation (SREM)
  arXiv  Detail & Related papers  (2023-12-27T09:03:43Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• NoiER: An Approach for Training more Reliable Fine-TunedDownstream Task Models [54.184609286094044]
  We propose noise entropy regularisation (NoiER) as an efficient learning paradigm that solves the problem without auxiliary models and additional data. The proposed approach improved traditional OOD detection evaluation metrics by 55% on average compared to the original fine-tuned models.
  arXiv  Detail & Related papers  (2021-08-29T06:58:28Z)
• Self-Damaging Contrastive Learning [92.34124578823977]
  Unlabeled data in reality is commonly imbalanced and shows a long-tail distribution. This paper proposes a principled framework called Self-Damaging Contrastive Learning to automatically balance the representation learning without knowing the classes. Our experiments show that SDCLR significantly improves not only overall accuracies but also balancedness.
  arXiv  Detail & Related papers  (2021-06-06T00:04:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.