Knowledge Distillation with Training Wheels

• URL: http://arxiv.org/abs/2502.17717v1
• Date: Mon, 24 Feb 2025 23:17:52 GMT
• Title: Knowledge Distillation with Training Wheels
• Authors: Guanlin Liu, Anand Ramachandran, Tanmay Gangwani, Yan Fu, Abhinav Sethy,
• Abstract summary: We formulate a more general framework for knowledge distillation where the student learns from the teacher during training.<n>We extend this using constrained reinforcement learning to a framework that incorporates the use of the teacher model as a test-time reference.
• Score: 15.153745235245287
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge distillation is used, in generative language modeling, to train a smaller student model using the help of a larger teacher model, resulting in improved capabilities for the student model. In this paper, we formulate a more general framework for knowledge distillation where the student learns from the teacher during training, and also learns to ask for the teacher's help at test-time following rules specifying test-time restrictions. Towards this, we first formulate knowledge distillation as an entropy-regularized value optimization problem. Adopting Path Consistency Learning to solve this, leads to a new knowledge distillation algorithm using on-policy and off-policy demonstrations. We extend this using constrained reinforcement learning to a framework that incorporates the use of the teacher model as a test-time reference, within constraints. In this situation, akin to a human learner, the model needs to learn not only the learning material, but also the relative difficulty of different sections to prioritize for seeking teacher help. We examine the efficacy of our method through experiments in translation and summarization tasks, observing trends in accuracy and teacher use, noting that our approach unlocks operating points not available to the popular Speculative Decoding approach.

Related papers

• UNDO: Understanding Distillation as Optimization [9.100811514331498]
  We introduce the UNDO: UNderstanding Distillation as Optimization framework. Each iteration directly targets the student's learning deficiencies, motivating the teacher to provide tailored and enhanced rationales. Empirical evaluations on various challenging mathematical and commonsense reasoning tasks demonstrate that our iterative distillation method, UNDO, significantly outperforms standard one-step distillation methods.
  arXiv  Detail & Related papers  (2025-04-03T12:18:51Z)
• Knowledge Graph Enhanced Generative Multi-modal Models for Class-Incremental Learning [51.0864247376786]
  We introduce a Knowledge Graph Enhanced Generative Multi-modal model (KG-GMM) that builds an evolving knowledge graph throughout the learning process. During testing, we propose a Knowledge Graph Augmented Inference method that locates specific categories by analyzing relationships within the generated text.
  arXiv  Detail & Related papers  (2025-03-24T07:20:43Z)
• When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets? [0.0]
  We present our submission to the BabyLM challenge, aiming to push the boundaries of data-efficient language model pretraining. We address the limitation of treating students equally by formulating weighted mutual learning as a bi-level optimization problem. Our evaluations show that teacher-less methods can match or surpass teacher-supervised approaches.
  arXiv  Detail & Related papers  (2024-11-25T15:25:31Z)
• Knowledge Distillation for Road Detection based on cross-model Semi-Supervised Learning [17.690698736544626]
  We propose an integrated approach that combines knowledge distillation and semi-supervised learning methods. This hybrid approach leverages the robust capabilities of large models to effectively utilise large unlabelled data. The proposed semi-supervised learning-based knowledge distillation (SSLKD) approach demonstrates a notable improvement in the performance of the student model.
  arXiv  Detail & Related papers  (2024-02-07T22:50:47Z)
• Teacher Agent: A Knowledge Distillation-Free Framework for Rehearsal-based Video Incremental Learning [29.52218286906986]
  Rehearsal-based video incremental learning often employs knowledge distillation to mitigate catastrophic forgetting of previously learned data. We propose a knowledge distillation-free framework for rehearsal-based video incremental learning called textitTeacher Agent.
  arXiv  Detail & Related papers  (2023-06-01T06:54:56Z)
• Better Teacher Better Student: Dynamic Prior Knowledge for Knowledge Distillation [70.92135839545314]
  We propose the dynamic prior knowledge (DPK), which integrates part of teacher's features as the prior knowledge before the feature distillation. Our DPK makes the performance of the student model positively correlated with that of the teacher model, which means that we can further boost the accuracy of students by applying larger teachers.
  arXiv  Detail & Related papers  (2022-06-13T11:52:13Z)
• On the benefits of knowledge distillation for adversarial robustness [53.41196727255314]
  We show that knowledge distillation can be used directly to boost the performance of state-of-the-art models in adversarial robustness. We present Adversarial Knowledge Distillation (AKD), a new framework to improve a model's robust performance.
  arXiv  Detail & Related papers  (2022-03-14T15:02:13Z)
• RLTutor: Reinforcement Learning Based Adaptive Tutoring System by Modeling Virtual Student with Fewer Interactions [10.34673089426247]
  We propose a framework for optimizing teaching strategies by constructing a virtual model of the student. Our results can serve as a buffer between theoretical instructional optimization and practical applications in e-learning systems.
  arXiv  Detail & Related papers  (2021-07-31T15:42:03Z)
• Learning Student-Friendly Teacher Networks for Knowledge Distillation [50.11640959363315]
  We propose a novel knowledge distillation approach to facilitate the transfer of dark knowledge from a teacher to a student. Contrary to most of the existing methods that rely on effective training of student models given pretrained teachers, we aim to learn the teacher models that are friendly to students.
  arXiv  Detail & Related papers  (2021-02-12T07:00:17Z)
• Learning to Reweight with Deep Interactions [104.68509759134878]
  We propose an improved data reweighting algorithm, in which the student model provides its internal states to the teacher model. Experiments on image classification with clean/noisy labels and neural machine translation empirically demonstrate that our algorithm makes significant improvement over previous methods.
  arXiv  Detail & Related papers  (2020-07-09T09:06:31Z)
• Dual Policy Distillation [58.43610940026261]
  Policy distillation, which transfers a teacher policy to a student policy, has achieved great success in challenging tasks of deep reinforcement learning. In this work, we introduce dual policy distillation(DPD), a student-student framework in which two learners operate on the same environment to explore different perspectives of the environment. The key challenge in developing this dual learning framework is to identify the beneficial knowledge from the peer learner for contemporary learning-based reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2020-06-07T06:49:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.