Interpretable Generative Adversarial Imitation Learning
- URL: http://arxiv.org/abs/2402.10310v1
- Date: Thu, 15 Feb 2024 20:21:40 GMT
- Title: Interpretable Generative Adversarial Imitation Learning
- Authors: Wenliang Liu, Danyang Li, Erfan Aasi, Roberto Tron, Calin Belta
- Abstract summary: We propose a novel imitation learning method that combines Signal Temporal Logic (STL) inference and control synthesis.
This approach not only provides a clear understanding of the task but also allows for the incorporation of human knowledge and adaptation to new scenarios.
- Score: 9.20323061622786
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Imitation learning methods have demonstrated considerable success in teaching
autonomous systems complex tasks through expert demonstrations. However, a
limitation of these methods is their lack of interpretability, particularly in
understanding the specific task the learning agent aims to accomplish. In this
paper, we propose a novel imitation learning method that combines Signal
Temporal Logic (STL) inference and control synthesis, enabling the explicit
representation of the task as an STL formula. This approach not only provides a
clear understanding of the task but also allows for the incorporation of human
knowledge and adaptation to new scenarios through manual adjustments of the STL
formulae. Additionally, we employ a Generative Adversarial Network
(GAN)-inspired training approach for both the inference and the control policy,
effectively narrowing the gap between the expert and learned policies. The
effectiveness of our algorithm is demonstrated through two case studies,
showcasing its practical applicability and adaptability.
Related papers
- REVEAL-IT: REinforcement learning with Visibility of Evolving Agent poLicy for InTerpretability [23.81322529587759]
REVEAL-IT is a novel framework for explaining the learning process of an agent in complex environments.
We visualize the policy structure and the agent's learning process for various training tasks.
A GNN-based explainer learns to highlight the most important section of the policy, providing a more clear and robust explanation of the agent's learning process.
arXiv Detail & Related papers (2024-06-20T11:29:26Z) - Data-CUBE: Data Curriculum for Instruction-based Sentence Representation
Learning [85.66907881270785]
We propose a data curriculum method, namely Data-CUBE, that arranges the orders of all the multi-task data for training.
In the task level, we aim to find the optimal task order to minimize the total cross-task interference risk.
In the instance level, we measure the difficulty of all instances per task, then divide them into the easy-to-difficult mini-batches for training.
arXiv Detail & Related papers (2024-01-07T18:12:20Z) - RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning.
Our proposed method uses reinforcement learning with user intervention signals themselves as rewards.
This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
arXiv Detail & Related papers (2023-11-21T21:05:21Z) - Resilient Constrained Learning [94.27081585149836]
This paper presents a constrained learning approach that adapts the requirements while simultaneously solving the learning task.
We call this approach resilient constrained learning after the term used to describe ecological systems that adapt to disruptions by modifying their operation.
arXiv Detail & Related papers (2023-06-04T18:14:18Z) - Task Phasing: Automated Curriculum Learning from Demonstrations [46.1680279122598]
Applying reinforcement learning to sparse reward domains is notoriously challenging due to insufficient guiding signals.
This paper introduces a principled task phasing approach that uses demonstrations to automatically generate a curriculum sequence.
Experimental results on 3 sparse reward domains demonstrate that our task phasing approaches outperform state-of-the-art approaches with respect to performance.
arXiv Detail & Related papers (2022-10-20T03:59:11Z) - Decoupled Adversarial Contrastive Learning for Self-supervised
Adversarial Robustness [69.39073806630583]
Adversarial training (AT) for robust representation learning and self-supervised learning (SSL) for unsupervised representation learning are two active research fields.
We propose a two-stage framework termed Decoupled Adversarial Contrastive Learning (DeACL)
arXiv Detail & Related papers (2022-07-22T06:30:44Z) - Delayed Reinforcement Learning by Imitation [31.932677462399468]
We present a novel algorithm that learns how to act in a delayed environment from undelayed demonstrations.
We show that DIDA obtains high performances with a remarkable sample efficiency on a variety of tasks.
arXiv Detail & Related papers (2022-05-11T15:27:33Z) - Shepherd Pre-trained Language Models to Develop a Train of Thought: An
Iterative Prompting Approach [30.117038793151004]
Pre-trained Language Models (PLMs) have been shown incapable of recalling knowledge to solve tasks requiring complex & multi-step inference procedures.
Similar to how humans develop a "train of thought" for these tasks, how can we equip PLMs with such abilities?
We propose an iterative context-aware prompter, which addresses these limitations by learning to dynamically synthesize conditioned prompts on the current step's contexts.
arXiv Detail & Related papers (2022-03-16T04:12:20Z) - Domain-Robust Visual Imitation Learning with Mutual Information
Constraints [0.0]
We introduce a new algorithm called Disentangling Generative Adversarial Imitation Learning (DisentanGAIL)
Our algorithm enables autonomous agents to learn directly from high dimensional observations of an expert performing a task.
arXiv Detail & Related papers (2021-03-08T21:18:58Z) - Off-Policy Imitation Learning from Observations [78.30794935265425]
Learning from Observations (LfO) is a practical reinforcement learning scenario from which many applications can benefit.
We propose a sample-efficient LfO approach that enables off-policy optimization in a principled manner.
Our approach is comparable with state-of-the-art locomotion in terms of both sample-efficiency and performance.
arXiv Detail & Related papers (2021-02-25T21:33:47Z) - Hierarchical Variational Imitation Learning of Control Programs [131.7671843857375]
We propose a variational inference method for imitation learning of a control policy represented by parametrized hierarchical procedures (PHP)
Our method discovers the hierarchical structure in a dataset of observation-action traces of teacher demonstrations, by learning an approximate posterior distribution over the latent sequence of procedure calls and terminations.
We demonstrate a novel benefit of variational inference in the context of hierarchical imitation learning: in decomposing the policy into simpler procedures, inference can leverage acausal information that is unused by other methods.
arXiv Detail & Related papers (2019-12-29T08:57:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.