STEP-OPD: Rethinking Output Targets and Internal Dynamics in On-Policy Distillation for Diffusion Models

• URL: http://arxiv.org/abs/2608.04887v1
• Date: Wed, 05 Aug 2026 14:11:27 GMT
• Title: STEP-OPD: Rethinking Output Targets and Internal Dynamics in On-Policy Distillation for Diffusion Models
• Abstract summary: We propose STEP-OPD, an on-policy distillation framework for image generation that extends the student's learning target beyond the teacher.<n>Instead of treating the teacher as the final target, we use the velocity difference between each task-specific teacher and the shared base model as a direction for further learning.<n>In addition, we align the direction and magnitude of representation changes between the student and teacher, enabling the student to learn how representations are progressively transformed.
• Score: 14.682283642975271
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: On-policy distillation (OPD) has become an effective approach for consolidating multiple task-specialized image generation models into a single student. However, existing OPD methods optimize the student mainly to match the teacher's output velocity, making the teacher the upper limit of the optimization objective. While output-level supervision alone leaves the student's blockwise representation evolution underconstrained, which weakens the transfer of capabilities that must be progressively developed across layers. We propose STEP-OPD, an on-policy distillation framework for image generation that extends the student's learning target beyond the teacher and introduces explicit constraints on its internal representation evolution. Instead of treating the teacher as the final target, we use the velocity difference between each task-specific teacher and the shared base model as a direction for further learning and add a scaled version of this difference to the teacher velocity. In addition, we align the direction and magnitude of representation changes between the student and teacher, enabling the student to learn how representations are progressively transformed across network blocks. Experiments on compositional alignment, text rendering, and human preference show that our method consistently improves Standard OPD methods. In particular, it increases the GenEval score of DiffusionOPD from 0.927 to 0.961, while also improving OCR and all preference-based metrics. The resulting unified student surpasses the corresponding single-task teachers across all three capability groups, showing that output extrapolation enables beyond-teacher learning. And representation change alignment provides complementary guidance for the student's internal transformations.

Related papers

• Poly-OPD: Heterogeneous Multi-Teacher On-Policy Distillation for Capability-Selectable Flow Models [25.503555874738623]
  Poly-OPD is a framework that can consolidate complementary strengths of heterogeneous teachers into a single compact flow-matching student.<n>Each student-generated image is re-encoded by a selected teacher's encoder and refined from a noise level matched by magnitude under the teacher's noise schedule.
  arXiv  Detail & Related papers  (2026-08-05T01:48:55Z)
• Distill What the Student Can See: Fisher-Projected On-Policy Distillation for Vision-Language Models [23.382646724389545]
  On-policy distillation (OPD) samples trajectories from the current student policy and minimizes token-level divergence between student and teacher next-token distributions.<n>We propose emphFisher-Projected On-Policy Distillation (FP-OPD), which distills only locally realizable teacher corrections.
  arXiv  Detail & Related papers  (2026-08-02T14:16:14Z)
• Contrastive On-Policy Distillation [36.47487086937924]
  We propose COPD, a contrastive On-policy Distillation framework.<n> COPD directly encourages the student model to learn more concise and efficient reasoning strategies.
  arXiv  Detail & Related papers  (2026-07-21T12:35:04Z)
• H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation [31.41588745047352]
  H-OPD is proposed as a confidence-aware heterogeneous multi-teacher OPD framework for multimodal reasoning.<n>By verifying the complementarity of heterogeneous teachers in the same reasoning process, H-OPD replaces task or sample level teacher routing.<n>H-OPD employs vision-to-language description transfer to enable text-only teachers to access key visual semantics.
  arXiv  Detail & Related papers  (2026-07-01T07:37:10Z)
• On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents [50.78688247330029]
  On-Policy Distillation (OPD) is a natural recipe for transferring such capabilities to smaller students.<n>We propose Guided On-Policy Distillation (Guided-OPD), a simple yet effective algorithm that mixes teacher- and student-generated turns within each rollout.<n>Guided-OPD improves Score by 21.1% and Success Rate by 25.5% over vanilla OPD on average.
  arXiv  Detail & Related papers  (2026-06-14T16:41:45Z)
• Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe [53.40076304466524]
  We propose Uni-OPD, a unified OPD framework that generalizes across Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs)<n>Specifically, from the student's perspective, we adopt two data balancing strategies to promote exploration of informative student-generated states during training.<n>We develop an outcome-guided margin calibration mechanism to restore order consistency between correct and incorrect trajectories.
  arXiv  Detail & Related papers  (2026-05-05T12:15:21Z)
• Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation [57.524909883706556]
  On-policy distillation (OPD) has demonstrated strong empirical gains in improving student performance.<n>This work introduces a flexible reference model and a reward scaling factor that controls the relative weight of the reward term against the KL regularization.<n>In particular, in the setting where we merge the knowledge from different domain experts, obtained by applying domain-specific RL to the same student model, ExOPD enables the student to even surpass the teacher's performance boundary.
  arXiv  Detail & Related papers  (2026-02-12T16:14:29Z)
• Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher [43.678380057638016]
  Gap Preserving Distillation (GPD) method trains an additional dynamic teacher model from scratch along with training the student to bridge this gap. In experiments, GPD significantly outperforms existing distillation methods on top of both CNNs and transformers architectures. GPD also generalizes well to the scenarios without a pre-trained teacher, including training from scratch and fine-tuning, yielding a large improvement of 1.80% and 0.89% on ResNet18.
  arXiv  Detail & Related papers  (2024-10-05T12:29:51Z)
• Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models [62.5501109475725]
  Knowledge distillation (KD) is a technique that compresses large teacher models by training smaller student models to mimic them. This paper introduces Online Knowledge Distillation (OKD), where the teacher network integrates small online modules to concurrently train with the student model. OKD achieves or exceeds the performance of leading methods in various model architectures and sizes, reducing training time by up to fourfold.
  arXiv  Detail & Related papers  (2024-09-19T07:05:26Z)
• Distantly-Supervised Named Entity Recognition with Adaptive Teacher Learning and Fine-grained Student Ensemble [56.705249154629264]
  Self-training teacher-student frameworks are proposed to improve the robustness of NER models. In this paper, we propose an adaptive teacher learning comprised of two teacher-student networks. Fine-grained student ensemble updates each fragment of the teacher model with a temporal moving average of the corresponding fragment of the student, which enhances consistent predictions on each model fragment against noise.
  arXiv  Detail & Related papers  (2022-12-13T12:14:09Z)
• Representation Consolidation for Training Expert Students [54.90754502493968]
  We show that a multi-head, multi-task distillation method is sufficient to consolidate representations from task-specific teacher(s) and improve downstream performance. Our method can also combine the representational knowledge of multiple teachers trained on one or multiple domains into a single model.
  arXiv  Detail & Related papers  (2021-07-16T17:58:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.