Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach

• URL: http://arxiv.org/abs/2002.02202v1
• Date: Thu, 6 Feb 2020 11:31:04 GMT
• Title: Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach
• Authors: Zeyue Xue, Shuang Luo, Chao Wu, Pan Zhou, Kaigui Bian and Wei Du
• Abstract summary: We propose a brand new solution to reuse experiences and transfer value functions among multiple students via model distillation. We also describe how to design an efficient communication protocol to exploit heterogeneous knowledge. Our proposed framework, namely Learning and Teaching Categorical Reinforcement, shows promising performance on stabilizing and accelerating learning progress.
• Score: 55.83558520598304
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Peer-to-peer knowledge transfer in distributed environments has emerged as a promising method since it could accelerate learning and improve team-wide performance without relying on pre-trained teachers in deep reinforcement learning. However, for traditional peer-to-peer methods such as action advising, they have encountered difficulties in how to efficiently expressed knowledge and advice. As a result, we propose a brand new solution to reuse experiences and transfer value functions among multiple students via model distillation. But it is still challenging to transfer Q-function directly since it is unstable and not bounded. To address this issue confronted with existing works, we adopt Categorical Deep Q-Network. We also describe how to design an efficient communication protocol to exploit heterogeneous knowledge among multiple distributed agents. Our proposed framework, namely Learning and Teaching Categorical Reinforcement (LTCR), shows promising performance on stabilizing and accelerating learning progress with improved team-wide reward in four typical experimental environments.

Related papers

• Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher [52.2926020848095]
  Federated learning is vulnerable to white-box attacks and struggles to adapt to heterogeneous clients. This paper proposes a selective knowledge sharing mechanism for FD, termed Selective-FD.
  arXiv  Detail & Related papers  (2023-04-04T12:04:19Z)
• Distillation from Heterogeneous Models for Top-K Recommendation [43.83625440616829]
  HetComp is a framework that guides the student model by transferring sequences of knowledge from teachers' trajectories. HetComp significantly improves the distillation quality and the generalization of the student model.
  arXiv  Detail & Related papers  (2023-03-02T10:23:50Z)
• Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation [31.294947552032088]
  Data-free Knowledge Distillation (DFKD) has gained popularity recently, with the fundamental idea of carrying out knowledge transfer from a Teacher to a Student neural network in the absence of training data. We propose a meta-learning inspired framework by treating the task of Knowledge-Acquisition (learning from newly generated samples) and Knowledge-Retention (retaining knowledge on previously met samples) as meta-train and meta-test.
  arXiv  Detail & Related papers  (2023-02-28T03:50:56Z)
• Online Continual Learning via the Knowledge Invariant and Spread-out Properties [4.109784267309124]
  Key challenge in continual learning is catastrophic forgetting. We propose a new method, named Online Continual Learning via the Knowledge Invariant and Spread-out Properties (OCLKISP) We empirically evaluate our proposed method on four popular benchmarks for continual learning: Split CIFAR 100, Split SVHN, Split CUB200 and Split Tiny-Image-Net.
  arXiv  Detail & Related papers  (2023-02-02T04:03:38Z)
• Continual Prompt Tuning for Dialog State Tracking [58.66412648276873]
  A desirable dialog system should be able to continually learn new skills without forgetting old ones. We present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks.
  arXiv  Detail & Related papers  (2022-03-13T13:22:41Z)
• Information Theoretic Representation Distillation [20.802135299032308]
  We forge an alternative connection between information theory and knowledge distillation using a recently proposed entropy-like functional. Our method achieves competitive performance to state-of-the-art on the knowledge distillation and cross-model transfer tasks. We shed light to a new state-of-the-art for binary quantisation.
  arXiv  Detail & Related papers  (2021-12-01T12:39:50Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Peer Collaborative Learning for Online Knowledge Distillation [69.29602103582782]
  Peer Collaborative Learning method integrates online ensembling and network collaboration into a unified framework. Experiments on CIFAR-10, CIFAR-100 and ImageNet show that the proposed method significantly improves the generalisation of various backbone networks.
  arXiv  Detail & Related papers  (2020-06-07T13:21:52Z)
• Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification [106.08067870620218]
  We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME) We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model. We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-06T12:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.