Complex Skill Acquisition Through Simple Skill Imitation Learning
- URL: http://arxiv.org/abs/2007.10281v4
- Date: Mon, 19 Oct 2020 19:43:49 GMT
- Title: Complex Skill Acquisition Through Simple Skill Imitation Learning
- Authors: Pranay Pasula
- Abstract summary: We propose a new algorithm that trains neural network policies on simple, easy-to-learn skills.
We focus on the case in which the complex task comprises a concurrent (and possibly sequential) combination of the simpler subtasks.
Our algorithm consistently outperforms a state-of-the-art baseline in training speed and overall performance.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Humans often think of complex tasks as combinations of simpler subtasks in
order to learn those complex tasks more efficiently. For example, a backflip
could be considered a combination of four subskills: jumping, tucking knees,
rolling backwards, and thrusting arms downwards. Motivated by this line of
reasoning, we propose a new algorithm that trains neural network policies on
simple, easy-to-learn skills in order to cultivate latent spaces that
accelerate imitation learning of complex, hard-to-learn skills. We focus on the
case in which the complex task comprises a concurrent (and possibly sequential)
combination of the simpler subtasks, and therefore our algorithm can be seen as
a novel approach to concurrent hierarchical imitation learning. We evaluate our
algorithm on difficult tasks in a high-dimensional environment and find that it
consistently outperforms a state-of-the-art baseline in training speed and
overall performance.
Related papers
- Reduce, Reuse, Recycle: Categories for Compositional Reinforcement Learning [19.821117942806474]
We view task composition through the prism of category theory.
The categorical properties of Markov decision processes untangle complex tasks into manageable sub-tasks.
Experimental results support the categorical theory of reinforcement learning.
arXiv Detail & Related papers (2024-08-23T21:23:22Z) - Fast Inference and Transfer of Compositional Task Structures for
Few-shot Task Generalization [101.72755769194677]
We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph.
Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks.
Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
arXiv Detail & Related papers (2022-05-25T10:44:25Z) - Divide & Conquer Imitation Learning [75.31752559017978]
Imitation Learning can be a powerful approach to bootstrap the learning process.
We present a novel algorithm designed to imitate complex robotic tasks from the states of an expert trajectory.
We show that our method imitates a non-holonomic navigation task and scales to a complex simulated robotic manipulation task with very high sample efficiency.
arXiv Detail & Related papers (2022-04-15T09:56:50Z) - Hierarchical Skills for Efficient Exploration [70.62309286348057]
In reinforcement learning, pre-trained low-level skills have the potential to greatly facilitate exploration.
Prior knowledge of the downstream task is required to strike the right balance between generality (fine-grained control) and specificity (faster learning) in skill design.
We propose a hierarchical skill learning framework that acquires skills of varying complexity in an unsupervised manner.
arXiv Detail & Related papers (2021-10-20T22:29:32Z) - Efficiently Identifying Task Groupings for Multi-Task Learning [55.80489920205404]
Multi-task learning can leverage information learned by one task to benefit the training of other tasks.
We suggest an approach to select which tasks should train together in multi-task learning models.
Our method determines task groupings in a single training run by co-training all tasks together and quantifying the effect to which one task's gradient would affect another task's loss.
arXiv Detail & Related papers (2021-09-10T02:01:43Z) - Reset-Free Reinforcement Learning via Multi-Task Learning: Learning
Dexterous Manipulation Behaviors without Human Intervention [67.1936055742498]
We show that multi-task learning can effectively scale reset-free learning schemes to much more complex problems.
This work shows the ability to learn dexterous manipulation behaviors in the real world with RL without any human intervention.
arXiv Detail & Related papers (2021-04-22T17:38:27Z) - One Network Fits All? Modular versus Monolithic Task Formulations in
Neural Networks [36.07011014271394]
We show that a single neural network is capable of simultaneously learning multiple tasks from a combined data set.
We study how the complexity of learning such combined tasks grows with the complexity of the task codes.
arXiv Detail & Related papers (2021-03-29T01:16:42Z) - Intrinsically Motivated Open-Ended Multi-Task Learning Using Transfer
Learning to Discover Task Hierarchy [0.0]
In open-ended continuous environments, robots need to learn multiple parameterised control tasks in hierarchical reinforcement learning.
We show that the most complex tasks can be learned more easily by transferring knowledge from simpler tasks, and faster by adapting the complexity of the actions to the task.
We propose a task-oriented representation of complex actions, called procedures, to learn online task relationships and unbounded sequences of action primitives to control the different observables of the environment.
arXiv Detail & Related papers (2021-02-19T10:44:08Z) - Active Hierarchical Imitation and Reinforcement Learning [0.0]
In this project, we explored different imitation learning algorithms and designed active learning algorithms upon the hierarchical imitation and reinforcement learning framework we have developed.
Our experimental results showed that using DAgger and reward-based active learning method can achieve better performance while saving more human efforts physically and mentally during the training process.
arXiv Detail & Related papers (2020-12-14T08:27:27Z) - Importance Weighted Policy Learning and Adaptation [89.46467771037054]
We study a complementary approach which is conceptually simple, general, modular and built on top of recent improvements in off-policy learning.
The framework is inspired by ideas from the probabilistic inference literature and combines robust off-policy learning with a behavior prior.
Our approach achieves competitive adaptation performance on hold-out tasks compared to meta reinforcement learning baselines and can scale to complex sparse-reward scenarios.
arXiv Detail & Related papers (2020-09-10T14:16:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.