On Neural Consolidation for Transfer in Reinforcement Learning

• URL: http://arxiv.org/abs/2210.02240v1
• Date: Wed, 5 Oct 2022 13:18:47 GMT
• Title: On Neural Consolidation for Transfer in Reinforcement Learning
• Authors: Valentin Guillet, Dennis G. Wilson, Carlos Aguilar-Melchor, Emmanuel Rachelson
• Abstract summary: We explore the use of network distillation as a feature extraction method to better understand the context in which transfer can occur. We show that distillation does not prevent knowledge transfer, including when transferring from multiple tasks to a new one, and we compare these results with transfer without prior distillation.
• Score: 4.129225533930966
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Although transfer learning is considered to be a milestone in deep reinforcement learning, the mechanisms behind it are still poorly understood. In particular, predicting if knowledge can be transferred between two given tasks is still an unresolved problem. In this work, we explore the use of network distillation as a feature extraction method to better understand the context in which transfer can occur. Notably, we show that distillation does not prevent knowledge transfer, including when transferring from multiple tasks to a new one, and we compare these results with transfer without prior distillation. We focus our work on the Atari benchmark due to the variability between different games, but also to their similarities in terms of visual features.

Related papers

• Disentangling and Mitigating the Impact of Task Similarity for Continual Learning [1.3597551064547502]
  Continual learning of partially similar tasks poses a challenge for artificial neural networks. High input feature similarity coupled with low readout similarity is catastrophic for both knowledge transfer and retention. Weight regularization based on the Fisher information metric significantly improves retention, regardless of task similarity.
  arXiv  Detail & Related papers  (2024-05-30T16:40:07Z)
• Evaluating the structure of cognitive tasks with transfer learning [67.22168759751541]
  This study investigates the transferability of deep learning representations between different EEG decoding tasks. We conduct extensive experiments using state-of-the-art decoding models on two recently released EEG datasets.
  arXiv  Detail & Related papers  (2023-07-28T14:51:09Z)
• Is forgetting less a good inductive bias for forward transfer? [7.704064306361941]
  We argue that the measure of forward transfer to a task should not be affected by the restrictions placed on the continual learner. Instead, forward transfer should be measured by how easy it is to learn a new task given a set of representations produced by continual learning on previous tasks. Our results indicate that less forgetful representations lead to a better forward transfer suggesting a strong correlation between retaining past information and learning efficiency on new tasks.
  arXiv  Detail & Related papers  (2023-03-14T19:52:09Z)
• Unsupervised Transfer Learning for Spatiotemporal Predictive Networks [90.67309545798224]
  We study how to transfer knowledge from a zoo of unsupervisedly learned models towards another network. Our motivation is that models are expected to understand complex dynamics from different sources. Our approach yields significant improvements on three benchmarks fortemporal prediction, and benefits the target even from less relevant ones.
  arXiv  Detail & Related papers  (2020-09-24T15:40:55Z)
• What is being transferred in transfer learning? [51.6991244438545]
  We show that when training from pre-trained weights, the model stays in the same basin in the loss landscape. We present that when training from pre-trained weights, the model stays in the same basin in the loss landscape and different instances of such model are similar in feature space and close in parameter space.
  arXiv  Detail & Related papers  (2020-08-26T17:23:40Z)
• Limits of Transfer Learning [0.0]
  We show the need to carefully select which sets of information to transfer and the need for dependence between transferred information and target problems. These results build on the algorithmic search framework for machine learning, allowing the results to apply to a wide range of learning problems using transfer.
  arXiv  Detail & Related papers  (2020-06-23T01:48:23Z)
• Knowledge Distillation Meets Self-Supervision [109.6400639148393]
  Knowledge distillation involves extracting "dark knowledge" from a teacher network to guide the learning of a student network. We show that the seemingly different self-supervision task can serve as a simple yet powerful solution. By exploiting the similarity between those self-supervision signals as an auxiliary task, one can effectively transfer the hidden information from the teacher to the student.
  arXiv  Detail & Related papers  (2020-06-12T12:18:52Z)
• Unsupervised Transfer Learning with Self-Supervised Remedy [60.315835711438936]
  Generalising deep networks to novel domains without manual labels is challenging to deep learning. Pre-learned knowledge does not transfer well without making strong assumptions about the learned and the novel domains. In this work, we aim to learn a discriminative latent space of the unlabelled target data in a novel domain by knowledge transfer from labelled related domains.
  arXiv  Detail & Related papers  (2020-06-08T16:42:17Z)
• Transfer Heterogeneous Knowledge Among Peer-to-Peer Teammates: A Model Distillation Approach [55.83558520598304]
  We propose a brand new solution to reuse experiences and transfer value functions among multiple students via model distillation. We also describe how to design an efficient communication protocol to exploit heterogeneous knowledge. Our proposed framework, namely Learning and Teaching Categorical Reinforcement, shows promising performance on stabilizing and accelerating learning progress.
  arXiv  Detail & Related papers  (2020-02-06T11:31:04Z)
• Inter- and Intra-domain Knowledge Transfer for Related Tasks in Deep Character Recognition [2.320417845168326]
  Pre-training a deep neural network on the ImageNet dataset is a common practice for training deep learning models. The technique of pre-training on one task and then retraining on a new one is called transfer learning. In this paper we analyse the effectiveness of using deep transfer learning for character recognition tasks.
  arXiv  Detail & Related papers  (2020-01-02T14:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.