Diverse Imitation Learning via Self-Organizing Generative Models

• URL: http://arxiv.org/abs/2205.03484v1
• Date: Fri, 6 May 2022 21:55:31 GMT
• Title: Diverse Imitation Learning via Self-Organizing Generative Models
• Authors: Arash Vahabpour, Tianyi Wang, Qiujing Lu, Omead Pooladzandi, Vwani Roychowdhury
• Abstract summary: Imitation learning is the task of replicating expert policy from demonstrations, without access to a reward function. We adopt an encoder-free generative model for behavior cloning (BC) to accurately distinguish and imitate different modes. We show that our method significantly outperforms the state of the art across multiple experiments.
• Score: 6.783186172518836
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Imitation learning is the task of replicating expert policy from demonstrations, without access to a reward function. This task becomes particularly challenging when the expert exhibits a mixture of behaviors. Prior work has introduced latent variables to model variations of the expert policy. However, our experiments show that the existing works do not exhibit appropriate imitation of individual modes. To tackle this problem, we adopt an encoder-free generative model for behavior cloning (BC) to accurately distinguish and imitate different modes. Then, we integrate it with GAIL to make the learning robust towards compounding errors at unseen states. We show that our method significantly outperforms the state of the art across multiple experiments.

Related papers

• Imitation from Diverse Behaviors: Wasserstein Quality Diversity Imitation Learning with Single-Step Archive Exploration [37.836675202590406]
  This work introduces Wasserstein Quality Diversity Imitation Learning (WQDIL) It improves the stability of imitation learning in the quality diversity setting with latent adversarial training based on a Wasserstein Auto-Encoder (WAE) It mitigates a behavior-overfitting issue using a measure-conditioned reward function with a single-step archive exploration bonus.
  arXiv  Detail & Related papers  (2024-11-11T13:11:18Z)
• IDIL: Imitation Learning of Intent-Driven Expert Behavior [2.07180164747172]
  We introduce IDIL, a novel imitation learning algorithm to mimic diverse intent-driven behaviors of experts. It is capable of addressing sequential tasks with high-dimensional state representations. As it creates a generative model, IDIL demonstrates superior performance in intent inference metrics.
  arXiv  Detail & Related papers  (2024-04-25T19:18:30Z)
• Sample-efficient Adversarial Imitation Learning [45.400080101596956]
  We propose a self-supervised representation-based adversarial imitation learning method to learn state and action representations. We show a 39% relative improvement over existing adversarial imitation learning methods on MuJoCo in a setting limited to 100 expert state-action pairs.
  arXiv  Detail & Related papers  (2023-03-14T12:36:01Z)
• Out-of-Dynamics Imitation Learning from Multimodal Demonstrations [68.46458026983409]
  We study out-of-dynamics imitation learning (OOD-IL), which relaxes the assumption to that the demonstrator and the imitator have the same state spaces. OOD-IL enables imitation learning to utilize demonstrations from a wide range of demonstrators but introduces a new challenge. We develop a better transferability measurement to tackle this newly-emerged challenge.
  arXiv  Detail & Related papers  (2022-11-13T07:45:06Z)
• Sequential Causal Imitation Learning with Unobserved Confounders [82.22545916247269]
  "Monkey see monkey do" is an age-old adage, referring to na"ive imitation without a deep understanding of a system's underlying mechanics. This paper investigates the problem of causal imitation learning in sequential settings, where the imitator must make multiple decisions per episode.
  arXiv  Detail & Related papers  (2022-08-12T13:53:23Z)
• Causal Imitation Learning with Unobserved Confounders [82.22545916247269]
  We study imitation learning when sensory inputs of the learner and the expert differ. We show that imitation could still be feasible by exploiting quantitative knowledge of the expert trajectories.
  arXiv  Detail & Related papers  (2022-08-12T13:29:53Z)
• Visual Adversarial Imitation Learning using Variational Models [60.69745540036375]
  Reward function specification remains a major impediment for learning behaviors through deep reinforcement learning. Visual demonstrations of desired behaviors often presents an easier and more natural way to teach agents. We develop a variational model-based adversarial imitation learning algorithm.
  arXiv  Detail & Related papers  (2021-07-16T00:15:18Z)
• Imitation by Predicting Observations [17.86983397979034]
  We present a new method for imitation solely from observations that achieves comparable performance to experts on challenging continuous control tasks. Our method, which we call FORM, is derived from an inverse RL objective and imitates using a model of expert behavior learned by generative modelling of the expert's observations. We show that FORM performs comparably to a strong baseline IRL method (GAIL) on the DeepMind Control Suite benchmark, while outperforming GAIL in the presence of task-irrelevant features.
  arXiv  Detail & Related papers  (2021-07-08T14:09:30Z)
• State-Only Imitation Learning for Dexterous Manipulation [63.03621861920732]
  In this paper, we explore state-only imitation learning. We train an inverse dynamics model and use it to predict actions for state-only demonstrations. Our method performs on par with state-action approaches and considerably outperforms RL alone.
  arXiv  Detail & Related papers  (2020-04-07T17:57:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.