RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learning

• URL: http://arxiv.org/abs/2105.03756v1
• Date: Sat, 8 May 2021 18:16:27 GMT
• Title: RAIL: A modular framework for Reinforcement-learning-based Adversarial Imitation Learning
• Authors: Eddy Hudson and Garrett Warnell and Peter Stone
• Abstract summary: We present an organizing, modular framework called Reinforcement-learning-based Adversarial Imitation Learning (RAIL) We create two new IfO (Imitation from Observation) algorithms, which we term SAIfO: SAC-based Adversarial Imitation from Observation and SILEM (Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch) In this paper, we focus on SAIfO, evaluating it on a suite of locomotion tasks from OpenAI Gym, and showing that it outperforms contemporaneous RAIL algorithms that perform IfO.
• Score: 47.535110066013736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Adversarial Imitation Learning (AIL) algorithms have recently led to state-of-the-art results on various imitation learning benchmarks, it is unclear as to what impact various design decisions have on performance. To this end, we present here an organizing, modular framework called Reinforcement-learning-based Adversarial Imitation Learning (RAIL) that encompasses and generalizes a popular subclass of existing AIL approaches. Using the view espoused by RAIL, we create two new IfO (Imitation from Observation) algorithms, which we term SAIfO: SAC-based Adversarial Imitation from Observation and SILEM (Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch). We go into greater depth about SILEM in a separate technical report. In this paper, we focus on SAIfO, evaluating it on a suite of locomotion tasks from OpenAI Gym, and showing that it outperforms contemporaneous RAIL algorithms that perform IfO.

Related papers

• AURORA: Augmented Understanding via Structured Reasoning and Reinforcement Learning for Reference Audio-Visual Segmentation [113.75682363364004]
  AURORA is a framework designed to enhance genuine reasoning and language comprehension in reference audio-visual segmentation.<n>AURORA achieves state-of-the-art performance on Ref-AVS benchmarks and generalizes effectively to unreferenced segmentation.
  arXiv  Detail & Related papers  (2025-08-04T07:47:38Z)
• Layered Unlearning for Adversarial Relearning [4.7066636827902]
  We study how post-training methods modify language model behavior and representations.<n>Recent results suggest that post-training induces shallow context-dependent circuits'' that suppress specific response patterns.<n>To test this hypothesis, we design an unlearning algorithm, Layered Unlearning (LU)<n>LU limits the ability of relearning on a subset of data to recover the full dataset.
  arXiv  Detail & Related papers  (2025-05-14T15:50:45Z)
• Adversarial Machine Unlearning [26.809123658470693]
  This paper focuses on the challenge of machine unlearning, aiming to remove the influence of specific training data on machine learning models. Traditionally, the development of unlearning algorithms runs parallel with that of membership inference attacks (MIA), a type of privacy threat. We propose a game-theoretic framework that integrates MIAs into the design of unlearning algorithms.
  arXiv  Detail & Related papers  (2024-06-11T20:07:22Z)
• USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval [115.28586222748478]
  Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality. Existing approaches typically suffer from two major limitations.
  arXiv  Detail & Related papers  (2023-01-17T12:42:58Z)
• An Empirical Investigation of Representation Learning for Imitation [76.48784376425911]
  Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data. We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation.
  arXiv  Detail & Related papers  (2022-05-16T11:23:42Z)
• Weakly Supervised Semantic Segmentation via Alternative Self-Dual Teaching [82.71578668091914]
  This paper establishes a compact learning framework that embeds the classification and mask-refinement components into a unified deep model. We propose a novel alternative self-dual teaching (ASDT) mechanism to encourage high-quality knowledge interaction.
  arXiv  Detail & Related papers  (2021-12-17T11:56:56Z)
• Skeletal Feature Compensation for Imitation Learning with Embodiment Mismatch [51.03498820458658]
  SILEM is a proposed imitation learning technique that compensates for differences in skeletal features obtained from the learner and expert. We create toy domains based on PyBullet's HalfCheetah and Ant to assess SILEM's benefits for this type of embodiment mismatch. We also provide qualitative and quantitative results on more realistic problems -- teaching simulated humanoid agents to walk by observing human demonstrations.
  arXiv  Detail & Related papers  (2021-04-15T22:50:48Z)
• Learn-Prune-Share for Lifelong Learning [25.678753894026357]
  We propose a learn-prune-share (LPS) algorithm which addresses the challenges of catastrophic forgetting, parsimony, and knowledge reuse simultaneously. LPS splits the network into task-specific partitions via an ADMM-based pruning strategy. This leads to no forgetting, while maintaining parsimony.
  arXiv  Detail & Related papers  (2020-12-13T04:05:16Z)
• Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration [21.870750931559915]
  We propose a new algorithm named Wasserstein Distance guided Adrial Imitation Learning (WDAIL) for promoting the performance of imitation learning (IL) The experiment results show that the learning procedure remains remarkably stable, and achieves significant performance in the complex continuous control tasks of MuJoCo.
  arXiv  Detail & Related papers  (2020-06-05T15:10:00Z)
• Provable Representation Learning for Imitation Learning via Bi-level Optimization [60.059520774789654]
  A common strategy in modern learning systems is to learn a representation that is useful for many tasks. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We instantiate this framework for the imitation learning settings of behavior cloning and observation-alone.
  arXiv  Detail & Related papers  (2020-02-24T21:03:52Z)
• Support-weighted Adversarial Imitation Learning [39.42395724783555]
  Adversarial Imitation Learning (AIL) is a family of imitation learning methods designed to mimic expert behaviors from demonstrations. We propose Support-weighted Adversarial Imitation Learning (SAIL), a general framework that extends a given AIL algorithm with information derived from support estimation of the expert policies. We show that the proposed method achieves better performance and training stability than baseline methods on a wide range of benchmark control tasks.
  arXiv  Detail & Related papers  (2020-02-20T15:34:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.