High-level Features for Resource Economy and Fast Learning in Skill Transfer

• URL: http://arxiv.org/abs/2106.10354v1
• Date: Fri, 18 Jun 2021 21:05:21 GMT
• Title: High-level Features for Resource Economy and Fast Learning in Skill Transfer
• Authors: Alper Ahmetoglu, Emre Ugur, Minoru Asada, Erhan Oztop
• Abstract summary: Deep networks are proven to be effective due to their ability to form increasingly complex abstractions. Previous work either enforced formation of abstractions creating a designer bias, or used a large number of neural units. We propose to exploit neural response dynamics to form compact representations to use in skill transfer.
• Score: 0.8602553195689513
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Abstraction is an important aspect of intelligence which enables agents to construct robust representations for effective decision making. In the last decade, deep networks are proven to be effective due to their ability to form increasingly complex abstractions. However, these abstractions are distributed over many neurons, making the re-use of a learned skill costly. Previous work either enforced formation of abstractions creating a designer bias, or used a large number of neural units without investigating how to obtain high-level features that may more effectively capture the source task. For avoiding designer bias and unsparing resource use, we propose to exploit neural response dynamics to form compact representations to use in skill transfer. For this, we consider two competing methods based on (1) maximum information compression principle and (2) the notion that abstract events tend to generate slowly changing signals, and apply them to the neural signals generated during task execution. To be concrete, in our simulation experiments, we either apply principal component analysis (PCA) or slow feature analysis (SFA) on the signals collected from the last hidden layer of a deep network while it performs a source task, and use these features for skill transfer in a new target task. We compare the generalization performance of these alternatives with the baselines of skill transfer with full layer output and no-transfer settings. Our results show that SFA units are the most successful for skill transfer. SFA as well as PCA, incur less resources compared to usual skill transfer, whereby many units formed show a localized response reflecting end-effector-obstacle-goal relations. Finally, SFA units with lowest eigenvalues resembles symbolic representations that highly correlate with high-level features such as joint angles which might be thought of precursors for fully symbolic systems.

Related papers

• Gradient-based inference of abstract task representations for generalization in neural networks [5.794537047184604]
  We show that gradients backpropagated through a neural network to a task representation layer are an efficient way to infer current task demands. We demonstrate that gradient-based inference provides higher learning efficiency and generalization to novel tasks and limits.
  arXiv  Detail & Related papers  (2024-07-24T15:28:08Z)
• Knowledge Transfer in Deep Reinforcement Learning via an RL-Specific GAN-Based Correspondence Function [0.0]
  This article introduces a novel approach that modifies Cycle Generative Adversarial Networks specifically for reinforcement learning. Our method achieves 100% knowledge transfer in identical tasks and either 100% knowledge transfer or a 30% reduction in training time for a rotated task.
  arXiv  Detail & Related papers  (2022-09-14T12:42:59Z)
• Learning Abstract and Transferable Representations for Planning [25.63560394067908]
  We propose a framework for autonomously learning state abstractions of an agent's environment. These abstractions are task-independent, and so can be reused to solve new tasks. We show how to combine these portable representations with problem-specific ones to generate a sound description of a specific task.
  arXiv  Detail & Related papers  (2022-05-04T14:40:04Z)
• INFOrmation Prioritization through EmPOWERment in Visual Model-Based RL [90.06845886194235]
  We propose a modified objective for model-based reinforcement learning (RL) We integrate a term inspired by variational empowerment into a state-space model based on mutual information. We evaluate the approach on a suite of vision-based robot control tasks with natural video backgrounds.
  arXiv  Detail & Related papers  (2022-04-18T23:09:23Z)
• Fractional Transfer Learning for Deep Model-Based Reinforcement Learning [0.966840768820136]
  Reinforcement learning (RL) is well known for requiring large amounts of data in order for RL agents to learn to perform complex tasks. Recent progress in model-based RL allows agents to be much more data-efficient. We present a simple alternative approach: fractional transfer learning.
  arXiv  Detail & Related papers  (2021-08-14T12:44:42Z)
• Hierarchical Few-Shot Imitation with Skill Transition Models [66.81252581083199]
  Few-shot Imitation with Skill Transition Models (FIST) is an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments.
  arXiv  Detail & Related papers  (2021-07-19T15:56:01Z)
• Emergent Symbols through Binding in External Memory [2.3562267625320352]
  We introduce the Emergent Symbol Binding Network (ESBN), a recurrent network augmented with an external memory. This binding mechanism allows symbol-like representations to emerge through the learning process without the need to explicitly incorporate symbol-processing machinery. Across a series of tasks, we show that this architecture displays nearly perfect generalization of learned rules to novel entities.
  arXiv  Detail & Related papers  (2020-12-29T04:28:32Z)
• Lightweight Single-Image Super-Resolution Network with Attentive Auxiliary Feature Learning [73.75457731689858]
  We develop a computation efficient yet accurate network based on the proposed attentive auxiliary features (A$2$F) for SISR. Experimental results on large-scale dataset demonstrate the effectiveness of the proposed model against the state-of-the-art (SOTA) SR methods.
  arXiv  Detail & Related papers  (2020-11-13T06:01:46Z)
• Towards Accurate Knowledge Transfer via Target-awareness Representation Disentanglement [56.40587594647692]
  We propose a novel transfer learning algorithm, introducing the idea of Target-awareness REpresentation Disentanglement (TRED) TRED disentangles the relevant knowledge with respect to the target task from the original source model and used as a regularizer during fine-tuning the target model. Experiments on various real world datasets show that our method stably improves the standard fine-tuning by more than 2% in average.
  arXiv  Detail & Related papers  (2020-10-16T17:45:08Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)
• Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment [84.57874289554839]
  Training deep neural networks on large-scale datasets requires significant hardware resources. Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize. We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
  arXiv  Detail & Related papers  (2020-02-10T16:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.