Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs

• URL: http://arxiv.org/abs/2609.02548v1
• Date: Wed, 02 Sep 2026 13:00:45 GMT
• Title: Learn from Whoever Is Right: Answer-Verified Multi-Teacher Distillation for Multi-Domain LLMs
• Abstract summary: Multi-Teacher Self-Distillation Policy Optimization (MT-SDPO) is an on-policy distillation method that unifies several frozen teachers into one student model.<n>MT-SDPO consists of three components: self-anchors, answer-verified eligibility, and privileged distillation.<n>It lifts the weakest domain of Qwen3-8B by 14.79 points and narrows its domain gap by 74.7%, a better balance than serving one matched teacher per domain.
• Score: 13.74158437063828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modern large language models (LLMs) rely on reinforcement learning to build strong capabilities in individual domains, but integrating those capabilities into a single deployable model remains challenging. By routing each sample to the teacher whose domain matches it, existing approaches let a domain label decide which teacher provides supervision. However, domain expertise holds only on average: the matched teacher is not always correct on a given sample, while a teacher from another domain sometimes is. The reliable teacher therefore has to be identified per sample, not per domain. In this paper, we introduce Multi-Teacher Self-Distillation Policy Optimization (MT-SDPO), an on-policy distillation method that unifies several frozen teachers into one student model. MT-SDPO consists of three components: (1) self-anchors, where a rollout is supervised by a correct rollout from its own group; (2) answer-verified eligibility, where a teacher may supervise a sample only if its own answer passes a verifier; and (3) privileged distillation, which merges the anchor and all verified feedback into one context that an exponential moving average self-teacher reads and the student does not, thereby keeping one policy at deployment. Across five students from three model families, MT-SDPO lifts the weakest domain of Qwen3-8B by 14.79 points and narrows its domain gap by 74.7%, a better balance than serving one matched teacher per domain. Verified reliability, not domain membership, should decide who teaches. Code is available at https://github.com/hexixiang/MT-SDPO.

Related papers

• Consolidating RLVR Capabilities Across Domains: A Deep Dive into Fusion Paradigms [50.28963615440865]
  We organize three fusion paradigms by the artefacts they reuse.<n>We compare all three using shared experts and data across model scales and a multi-domain benchmark suite.
  arXiv  Detail & Related papers  (2026-08-27T17:38:04Z)
• D$^3$-MOPD: Adaptive Dynamic Domain ScheDuling for Efficient Multi-Teacher Distillation [50.89072318748398]
  Multi-teacher on-policy distillation distills several domain-expert teachers into a single student.<n>Existing approaches typically fix the per-domain data mixture before training.<n>A fixed mixture wastes compute on fast-converging domains and undertrains slower-converging ones.<n>We propose D$3$-MOPD, a zero-overhead scheduler that repurposes the per-domain reverse-KL signal already produced during training to adapt the domain mixture online.
  arXiv  Detail & Related papers  (2026-08-25T17:57:11Z)
• OPOD: On-Policy Omni Distillation [55.440058537827134]
  Training a single model on pooled multimodal data often fails to match models specialized for individual modalities.<n>We present On-Policy Omni Distillation (OPD), which routes each student response to the matching text, image, or audio teacher.<n>OPD achieves the best average score at every scale, reaching 70.8, 51.7, and 46.2.
  arXiv  Detail & Related papers  (2026-07-23T04:55:04Z)
• MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training [29.71786876690702]
  Multi-teacher On-Policy Distillation (MOPD) is a post-training paradigm for combining the capabilities of multiple domain RL teachers.<n>On Qwen3-30B-A3B, MOPD outperforms Mix-RL, Cascade RL, Off-Policy Finetune, and Param-Merge baselines.
  arXiv  Detail & Related papers  (2026-06-29T14:51:28Z)
• Following the Teacher's Footsteps: Scheduled Checkpoint Distillation for Domain-Specific LLMs [5.786917616876281]
  Large language models (LLMs) are challenging to deploy for domain-specific tasks due to their massive scale.<n>While distilling a finetuned LLM into a smaller student model is a promising alternative, the capacity gap between teacher and student often leads to suboptimal performance.<n>We propose a novel theoretical insight: a student can outperform its teacher if its advantage on a Student-Favored Subdomain outweighs its deficit on the Teacher-Favored Subdomain.
  arXiv  Detail & Related papers  (2026-01-15T06:46:01Z)
• ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning [95.78635058475439]
  Cross-Domain Few-Shot Learning aims at addressing the Few-Shot Learning problem across different domains. This paper technically contributes a novel Multi-Expert Domain Decompositional Network (ME-D2N) We present a novel domain decomposition module that learns to decompose the student model into two domain-related sub parts.
  arXiv  Detail & Related papers  (2022-10-11T09:24:47Z)
• Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts [33.21435044949033]
  Most existing methods perform training on multiple source domains using a single model. We propose a novel framework for unsupervised test-time adaptation, which is formulated as a knowledge distillation process.
  arXiv  Detail & Related papers  (2022-10-08T02:28:10Z)
• Representation Consolidation for Training Expert Students [54.90754502493968]
  We show that a multi-head, multi-task distillation method is sufficient to consolidate representations from task-specific teacher(s) and improve downstream performance. Our method can also combine the representational knowledge of multiple teachers trained on one or multiple domains into a single model.
  arXiv  Detail & Related papers  (2021-07-16T17:58:18Z)
• One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers [54.146208195806636]
  We propose a multi-teacher knowledge distillation framework named MT-BERT for pre-trained language model compression. We show that MT-BERT can train high-quality student model from multiple teacher PLMs. Experiments on three benchmark datasets validate the effectiveness of MT-BERT in compressing PLMs.
  arXiv  Detail & Related papers  (2021-06-02T08:42:33Z)
• Unsupervised Multi-Target Domain Adaptation Through Knowledge Distillation [14.088776449829345]
  Unsupervised domain adaptation (UDA) seeks to alleviate the problem of domain shift between the distribution of unlabeled data. In this paper, we propose a novel unsupervised MTDA approach to train a CNN that can generalize well across multiple target domains.
  arXiv  Detail & Related papers  (2020-07-14T14:59:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.