D$^3$-MOPD: Adaptive Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation

• URL: http://arxiv.org/abs/2608.24987v1
• Date: Tue, 25 Aug 2026 17:57:11 GMT
• Title: D$^3$-MOPD: Adaptive Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation
• Abstract summary: Multi-teacher on-policy distillation distills several domain-expert teachers into a single student.<n>Existing approaches typically fix the per-domain data mixture before training.<n>A fixed mixture wastes compute on fast-converging domains and undertrains slower-converging ones.<n>We propose D$3$-MOPD, a zero-overhead scheduler that repurposes the per-domain reverse-KL signal already produced during training to adapt the domain mixture online.
• Score: 50.89072318748398
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Multi-teacher on-policy distillation (MOPD) distills several domain-expert teachers into a single student by minimizing per-domain reverse-KL divergence on the student's own rollouts. Existing approaches typically fix the per-domain data mixture before training, overlooking the fact that different domains converge at substantially different rates: some plateau early while others continue to improve throughout the training budget. A fixed mixture therefore wastes compute on fast-converging domains and undertrains slower-converging ones. To address this, we propose D$^3$-MOPD (Dynamic Domain ScheDuling for MOPD), a zero-overhead scheduler that repurposes the per-domain reverse-KL signal already produced during training to adapt the domain mixture online. Running asynchronously outside the training process, an off-process watcher periodically tracks each domain's KL trajectory, estimates remaining headroom and current improvement rate, and accordingly adjusts the domain sampling ratios without altering the core training loop. Our D$^3$-MOPD scales naturally to arbitrary numbers of domains, and the expected benefit grows as more domains introduce more diverse convergence patterns for the scheduler to exploit. On a Qwen3.6-35B-A3B student distilled from four domain-expert teachers, D$^3$-MOPD closes 97% of the average student-to-teacher performance gap, compared with 63% for vanilla MOPD, reaches the same peak performance with an approximately 3$\times$ reduction in rollout steps, and surpasses the specialist teachers on three of seven benchmarks.

Related papers

• Know When to Stop, Where to Restart: Accelerating Multi-Turn Agentic On-Policy Distillation [45.25745441009372]
  Existing acceleration methods truncate or relocate the supervision signal according to fixed, offline budgets.<n>We propose STRIDE (Stop-and-Restart on-policy Distillation acceleration), which combines two complementary techniques.<n>Our method matches full-trajectory OPD and exceeds the 30B teacher at a $3.73times$ speedup, surpasses the baseline itself at $2.34times$, and retains a $4.51times$ speedup under cross-domain multi-teacher training.
  arXiv  Detail & Related papers  (2026-09-13T16:17:35Z)
• Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs [13.74158437063828]
  Multi-Teacher Self-Distillation Policy Optimization (MT-SDPO) is an on-policy distillation method that unifies several frozen teachers into one student model.<n>MT-SDPO consists of three components: self-anchors, answer-verified eligibility, and privileged distillation.<n>It lifts the weakest domain of Qwen3-8B by 14.79 points and narrows its domain gap by 74.7%, a better balance than serving one matched teacher per domain.
  arXiv  Detail & Related papers  (2026-09-02T13:00:45Z)
• Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms [50.28963615440865]
  We organize three fusion paradigms by the artefacts they reuse.<n>We compare all three using shared experts and data across model scales and a multi-domain benchmark suite.
  arXiv  Detail & Related papers  (2026-08-27T17:38:04Z)
• MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training [29.71786876690702]
  Multi-teacher On-Policy Distillation (MOPD) is a post-training paradigm for combining the capabilities of multiple domain RL teachers.<n>On Qwen3-30B-A3B, MOPD outperforms Mix-RL, Cascade RL, Off-Policy Finetune, and Param-Merge baselines.
  arXiv  Detail & Related papers  (2026-06-29T14:51:28Z)
• Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR [59.527728403519745]
  Transfer-Aware Curriculum prioritizes domains whose updates broadly benefit the rest of the training suite.<n>It achieves the best macro-averaged accuracy on both Qwen3-1.7B and Llama3.2-3B.<n>It remains robust on imbalanced training mixtures where learnability-only curricula over-commit to dominant domains.
  arXiv  Detail & Related papers  (2026-06-23T21:10:29Z)
• A Local Perturbation Theory for Cross-Domain Interference and Recovery in Multi-Domain RL [49.589717630468876]
  We show that single-domain RL produces sparse, small-magnitude parameter edits with weak overlap among top-changed neurons.<n>We prove under a local model of multi-domain RL that later-domain training harms an earlier domain mainly through a second-order damage term.
  arXiv  Detail & Related papers  (2026-06-01T15:44:56Z)
• Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation [35.38408768980583]
  We propose DES-MoE, a dynamic expert specialization framework for multi-domain adaptation of Mixture-of-Experts models.<n> DES-MoE addresses catastrophic forgetting through three innovations: (1) an adaptive router balancing pre-trained knowledge retention and task-specific updates via distillation, (2) real-time expert-domain correlation mapping, and (3) a three-phase adaptive fine-tuning schedule that progressively freezes non-specialized parameters.
  arXiv  Detail & Related papers  (2025-09-21T02:30:04Z)
• D3T: Distinctive Dual-Domain Teacher Zigzagging Across RGB-Thermal Gap for Domain-Adaptive Object Detection [15.071470389431672]
  Domain adaptation for object detection typically entails transferring knowledge from one visible domain to another visible domain. We propose a Distinctive Dual-Domain Teacher (D3T) framework that employs distinct training paradigms for each domain. We validate our method through newly designed experimental protocols with well-known thermal datasets.
  arXiv  Detail & Related papers  (2024-03-14T13:05:43Z)
• Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning [20.17925272562433]
  Multi-domain learning aims to train a model with minimal average risk across multiple overlapping but non-identical domains. We propose Decoupled Training (D-Train) as a frustratingly easy and hyper parameter-free multi-domain learning method. D-Train is a tri-phase general-to-specific training strategy that first pre-trains on all domains to warm up a root model, then post-trains on each domain by splitting into multi-heads, and finally fine-tunes the heads by fixing the backbone.
  arXiv  Detail & Related papers  (2023-09-19T04:06:41Z)
• Improving Domain Generalization with Domain Relations [77.63345406973097]
  This paper focuses on domain shifts, which occur when the model is applied to new domains that are different from the ones it was trained on. We propose a new approach called D$3$G to learn domain-specific models. Our results show that D$3$G consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-02-06T08:11:16Z)
• Forget Less, Count Better: A Domain-Incremental Self-Distillation Learning Benchmark for Lifelong Crowd Counting [51.44987756859706]
  Off-the-shelf methods have some drawbacks to handle multiple domains. Lifelong Crowd Counting aims at alleviating the catastrophic forgetting and improving the generalization ability.
  arXiv  Detail & Related papers  (2022-05-06T15:37:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.