Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning

• URL: http://arxiv.org/abs/2004.08051v3
• Date: Thu, 8 Apr 2021 19:52:37 GMT
• Title: Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning
• Authors: Keuntaek Lee, Bogdan Vlahov, Jason Gibson, James M. Rehg, Evangelos A. Theodorou
• Abstract summary: We present a method for obtaining an implicit objective function for vision-based navigation. The proposed methodology relies on Imitation Learning, Model Predictive Control (MPC), and an interpretation technique used in Deep Neural Networks.
• Score: 34.5366377122507
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we present a method for obtaining an implicit objective function for vision-based navigation. The proposed methodology relies on Imitation Learning, Model Predictive Control (MPC), and an interpretation technique used in Deep Neural Networks. We use Imitation Learning as a means to do Inverse Reinforcement Learning in order to create an approximate cost function generator for a visual navigation challenge. The resulting cost function, the costmap, is used in conjunction with MPC for real-time control and outperforms other state-of-the-art costmap generators in novel environments. The proposed process allows for simple training and robustness to out-of-sample data. We apply our method to the task of vision-based autonomous driving in multiple real and simulated environments and show its generalizability.

Related papers

• Unsupervised Meta-Learning via In-Context Learning [3.4165401459803335]
  We propose a novel approach to unsupervised meta-learning that leverages the generalization abilities of in-supervised learning. Our method reframes meta-learning as a sequence modeling problem, enabling the transformer encoder to learn task context from support images.
  arXiv  Detail & Related papers  (2024-05-25T08:29:46Z)
• Graphical Object-Centric Actor-Critic [55.2480439325792]
  We propose a novel object-centric reinforcement learning algorithm combining actor-critic and model-based approaches. We use a transformer encoder to extract object representations and graph neural networks to approximate the dynamics of an environment. Our algorithm performs better in a visually complex 3D robotic environment and a 2D environment with compositional structure than the state-of-the-art model-free actor-critic algorithm.
  arXiv  Detail & Related papers  (2023-10-26T06:05:12Z)
• CoopInit: Initializing Generative Adversarial Networks via Cooperative Learning [50.90384817689249]
  CoopInit is a cooperative learning-based strategy that can quickly learn a good starting point for GANs. We demonstrate the effectiveness of the proposed approach on image generation and one-sided unpaired image-to-image translation tasks.
  arXiv  Detail & Related papers  (2023-03-21T07:49:32Z)
• Predictive Experience Replay for Continual Visual Control and Forecasting [62.06183102362871]
  We present a new continual learning approach for visual dynamics modeling and explore its efficacy in visual control and forecasting. We first propose the mixture world model that learns task-specific dynamics priors with a mixture of Gaussians, and then introduce a new training strategy to overcome catastrophic forgetting. Our model remarkably outperforms the naive combinations of existing continual learning and visual RL algorithms on DeepMind Control and Meta-World benchmarks with continual visual control tasks.
  arXiv  Detail & Related papers  (2023-03-12T05:08:03Z)
• Goal-Guided Transformer-Enabled Reinforcement Learning for Efficient Autonomous Navigation [15.501449762687148]
  We present a Goal-guided Transformer-enabled reinforcement learning (GTRL) approach for goal-driven navigation. Our approach motivates the scene representation to concentrate mainly on goal-relevant features, which substantially enhances the data efficiency of the DRL learning process. Both simulation and real-world experimental results manifest the superiority of our approach in terms of data efficiency, performance, robustness, and sim-to-real generalization.
  arXiv  Detail & Related papers  (2023-01-01T07:14:30Z)
• Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration [83.96729205383501]
  We introduce prompt-based learning to achieve fast adaptation for language embeddings. Our model can adapt to diverse vision-language navigation tasks, including VLN and REVERIE.
  arXiv  Detail & Related papers  (2022-03-08T11:01:24Z)
• Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learning [27.243603228431564]
  We propose a new IRLRL algorithm that learns a goal-conditionedtemporal reward function. The resulting costmap is used by Model Predictive Controllers (MPCs) to perform a task.
  arXiv  Detail & Related papers  (2022-01-17T17:36:29Z)
• Multitask Adaptation by Retrospective Exploration with Learned World Models [77.34726150561087]
  We propose a meta-learned addressing model called RAMa that provides training samples for the MBRL agent taken from task-agnostic storage. The model is trained to maximize the expected agent's performance by selecting promising trajectories solving prior tasks from the storage.
  arXiv  Detail & Related papers  (2021-10-25T20:02:57Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Vision-Based Autonomous Drone Control using Supervised Learning in Simulation [0.0]
  We propose a vision-based control approach using Supervised Learning for autonomous navigation and landing of MAVs in indoor environments. We trained a Convolutional Neural Network (CNN) that maps low resolution image and sensor input to high-level control commands. Our approach requires shorter training times than similar Reinforcement Learning approaches and can potentially overcome the limitations of manual data collection faced by comparable Supervised Learning approaches.
  arXiv  Detail & Related papers  (2020-09-09T13:45:41Z)
• Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude Estimation [0.0]
  This paper proposes a learning method for denoising gyroscopes of Inertial Measurement Units (IMUs) using ground truth data. The obtained algorithm outperforms the state-of-the-art on the (unseen) test sequences.
  arXiv  Detail & Related papers  (2020-02-25T08:04:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.