UniSD: Towards a Unified Self-Distillation Framework for Large Language Models

• URL: http://arxiv.org/abs/2605.06597v1
• Date: Thu, 07 May 2026 17:22:11 GMT
• Title: UniSD: Towards a Unified Self-Distillation Framework for Large Language Models
• Abstract summary: Self-distillation offers a promising path for adapting large language models (LLMs) without relying on stronger external teachers.<n>Existing methods mainly examine isolated design choices, leaving their effectiveness, roles, and interactions unclear.<n>We propose UniSD, a unified framework to systematically study self-distillation.
• Score: 38.12407816591177
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Self-distillation (SD) offers a promising path for adapting large language models (LLMs) without relying on stronger external teachers. However, SD in autoregressive LLMs remains challenging because self-generated trajectories are free-form, correctness is task-dependent, and plausible rationales can still provide unstable or unreliable supervision. Existing methods mainly examine isolated design choices, leaving their effectiveness, roles, and interactions unclear. In this paper, we propose UniSD, a unified framework to systematically study self-distillation. UniSD integrates complementary mechanisms that address supervision reliability, representation alignment, and training stability, including multi-teacher agreement, EMA teacher stabilization, token-level contrastive learning, feature matching, and divergence clipping. Across six benchmarks and six models from three model families, UniSD reveals when self-distillation improves over static imitation, which components drive the gains, and how these components interact across tasks. Guided by these insights, we construct UniSDfull, an integrated pipeline that combines complementary components and achieves the strongest overall performance, improving over the base model by +5.4 points and the strongest baseline by +2.8 points. Extensive evaluation highlights self-distillation as a practical and steerable approach for efficient LLM adaptation without stronger external teachers.

Related papers

• From Scoring to Acting: Outcome-Verified Comparative Self-Distillation for LLM Agents [16.67357658768852]
  We propose Outcome-Verified Comparative Self-Distillation (OVCSD)<n>OVCSD organizes failed student rollouts into a prefix tree, adaptively invokes a skill-conditioned teacher from student-reached states, and retains only outcome-verified successful continuations.<n> Experiments on ALFWorld and WebShop across three model scales show that OVCSD consistently outperforms skill-free RL and existing self-distillation baselines.
  arXiv  Detail & Related papers  (2026-07-30T09:47:58Z)
• Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillation [71.16738415436458]
  We introduce Reflection-Enhanced Self-Distillation (RESD), a framework that transforms raw failure feedback into an active source of corrective supervision.<n>RESD interprets failed trajectories by generating retrospective reflections to diagnose local errors, and curates a persistent global playbook.<n> Empirical evaluations on multiple continual learning tasks demonstrate that RESD substantially outperforms standard self-distillation baselines.
  arXiv  Detail & Related papers  (2026-05-12T20:46:05Z)
• SEIF: Self-Evolving Reinforcement Learning for Instruction Following [53.280277743734096]
  We propose SEIF (Self-Evolving Reinforcement Learning for Instruction Following), a self-evolving framework for enhancing the instruction-following ability of large language models (LLMs)<n>SEIF forms a closed self-evolution loop that improves the model's instruction-following ability, where instruction difficulty evolution and model capability evolution reinforce each other.<n> Experiments across multiple model scales and architectures show that SEIF consistently improves instruction-following performance, suggesting strong generality.
  arXiv  Detail & Related papers  (2026-05-08T09:13:12Z)
• Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe [53.40076304466524]
  We propose Uni-OPD, a unified OPD framework that generalizes across Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs)<n>Specifically, from the student's perspective, we adopt two data balancing strategies to promote exploration of informative student-generated states during training.<n>We develop an outcome-guided margin calibration mechanism to restore order consistency between correct and incorrect trajectories.
  arXiv  Detail & Related papers  (2026-05-05T12:15:21Z)
• "The Whole Is Greater Than the Sum of Its Parts": A Compatibility-Aware Multi-Teacher CoT Distillation Framework [16.96094045628127]
  Chain-of-Thought (CoT) reasoning empowers Large Language Models (LLMs) with remarkable capabilities but typically requires prohibitive parameter scales.<n>CoT distillation has emerged as a promising paradigm to transfer reasoning prowess into compact Student Models (SLMs)<n>We introduce COMPACT, a framework that adaptively fuses supervisions from different teachers by dynamically weighting teacher gradients.
  arXiv  Detail & Related papers  (2026-01-20T14:05:19Z)
• UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision [34.575729271291436]
  We formalize Conduction Aphasia, a phenomenon where models accurately interpret multimodal inputs but struggle to translate that understanding into faithful and controllable synthesis.<n>We propose UniCorn, a simple yet elegant self-improvement framework that eliminates the need for external data or teacher supervision.<n>To validate the restoration of multimodal coherence, we introduce UniCycle, a cycle-consistency benchmark based on a Text to Image to Text reconstruction loop.
  arXiv  Detail & Related papers  (2026-01-06T17:15:50Z)
• Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains [16.357026482329232]
  Multimodal LLMs produce fluent yet unreliable reasoning.<n>We introduce SR-MCR, a lightweight and label-free framework that aligns reasoning.<n> SR-MCR improves both answer accuracy and reasoning coherence across a broad set of visual benchmarks.
  arXiv  Detail & Related papers  (2025-12-27T10:14:14Z)
• MMT-ARD: Multimodal Multi-Teacher Adversarial Distillation for Robust Vision-Language Models [123.90007730845876]
  We propose MMT-ARD: a Multimodal Multi-Teacher Adversarial Distillation framework.<n>Our key innovation is a dual-teacher knowledge fusion architecture that collaboratively optimize clean feature preservation and robust feature enhancement.<n>Experiments on ImageNet and zero-shot benchmarks demonstrate that MMT-ARD improves robust accuracy by +4.32% and zero-shot accuracy by +3.5%.
  arXiv  Detail & Related papers  (2025-11-21T17:46:44Z)
• AdaSwitch: Adaptive Switching Generation for Knowledge Distillation [58.647880811071495]
  Small language models (SLMs) are crucial for applications with strict latency and computational constraints.<n>We propose AdaSwitch, a novel approach that combines on-policy and off-policy generation at the token level.<n>AdaSwitch consistently improves accuracy, offering a practical and effective method for distilling SLMs with acceptable additional overhead.
  arXiv  Detail & Related papers  (2025-10-09T06:38:37Z)
• SELF: Self-Evolution with Language Feedback [68.6673019284853]
  'SELF' (Self-Evolution with Language Feedback) is a novel approach to advance large language models. It enables LLMs to self-improve through self-reflection, akin to human learning processes. Our experiments in mathematics and general tasks demonstrate that SELF can enhance the capabilities of LLMs without human intervention.
  arXiv  Detail & Related papers  (2023-10-01T00:52:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.