Prioritized Experience-based Reinforcement Learning with Human Guidance: Methdology and Application to Autonomous Driving

• URL: http://arxiv.org/abs/2109.12516v1
• Date: Sun, 26 Sep 2021 07:19:26 GMT
• Title: Prioritized Experience-based Reinforcement Learning with Human Guidance: Methdology and Application to Autonomous Driving
• Authors: Jingda Wu, Zhiyu Huang, Wenhui Huang, Chen Lv
• Abstract summary: Reinforcement learning requires skillful definition and remarkable computational efforts to solve optimization and control problems. In this paper, a comprehensive human guidance-based reinforcement learning framework is established. A novel prioritized experience replay mechanism that adapts to human guidance is proposed to boost the efficiency and performance of the reinforcement learning algorithm.
• Score: 2.5895890901896124
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reinforcement learning requires skillful definition and remarkable computational efforts to solve optimization and control problems, which could impair its prospect. Introducing human guidance into reinforcement learning is a promising way to improve learning performance. In this paper, a comprehensive human guidance-based reinforcement learning framework is established. A novel prioritized experience replay mechanism that adapts to human guidance in the reinforcement learning process is proposed to boost the efficiency and performance of the reinforcement learning algorithm. To relieve the heavy workload on human participants, a behavior model is established based on an incremental online learning method to mimic human actions. We design two challenging autonomous driving tasks for evaluating the proposed algorithm. Experiments are conducted to access the training and testing performance and learning mechanism of the proposed algorithm. Comparative results against the state-of-the-arts suggest the advantages of our algorithm in terms of learning efficiency, performance, and robustness.

Related papers

• Unveiling the Role of Expert Guidance: A Comparative Analysis of User-centered Imitation Learning and Traditional Reinforcement Learning [0.0]
  This study explores the performance, robustness, and limitations of imitation learning compared to traditional reinforcement learning methods. The insights gained from this study contribute to the advancement of human-centered artificial intelligence.
  arXiv  Detail & Related papers  (2024-10-28T18:07:44Z)
• GUIDE: Real-Time Human-Shaped Agents [4.676987516944155]
  We introduce GUIDE, a framework for real-time human-guided reinforcement learning. With only 10 minutes of human feedback, our algorithm achieves up to 30% increase in success rate compared to its RL baseline.
  arXiv  Detail & Related papers  (2024-10-19T18:59:39Z)
• RLIF: Interactive Imitation Learning as Reinforcement Learning [56.997263135104504]
  We show how off-policy reinforcement learning can enable improved performance under assumptions that are similar but potentially even more practical than those of interactive imitation learning. Our proposed method uses reinforcement learning with user intervention signals themselves as rewards. This relaxes the assumption that intervening experts in interactive imitation learning should be near-optimal and enables the algorithm to learn behaviors that improve over the potential suboptimal human expert.
  arXiv  Detail & Related papers  (2023-11-21T21:05:21Z)
• Human Decision Makings on Curriculum Reinforcement Learning with Difficulty Adjustment [52.07473934146584]
  We guide the curriculum reinforcement learning results towards a preferred performance level that is neither too hard nor too easy via learning from the human decision process. Our system is highly parallelizable, making it possible for a human to train large-scale reinforcement learning applications. It shows reinforcement learning performance can successfully adjust in sync with the human desired difficulty level.
  arXiv  Detail & Related papers  (2022-08-04T23:53:51Z)
• Teachable Reinforcement Learning via Advice Distillation [161.43457947665073]
  We propose a new supervision paradigm for interactive learning based on "teachable" decision-making systems that learn from structured advice provided by an external teacher. We show that agents that learn from advice can acquire new skills with significantly less human supervision than standard reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2022-03-19T03:22:57Z)
• Rethinking Learning Dynamics in RL using Adversarial Networks [79.56118674435844]
  We present a learning mechanism for reinforcement learning of closely related skills parameterized via a skill embedding space. The main contribution of our work is to formulate an adversarial training regime for reinforcement learning with the help of entropy-regularized policy gradient formulation.
  arXiv  Detail & Related papers  (2022-01-27T19:51:09Z)
• Behavior-based Neuroevolutionary Training in Reinforcement Learning [3.686320043830301]
  This work presents a hybrid algorithm that combines neuroevolutionary optimization with value-based reinforcement learning. For this purpose, we consolidate different methods to generate and optimize agent policies, creating a diverse population. Our results indicate that combining methods can enhance the sample efficiency and learning speed for evolutionary approaches.
  arXiv  Detail & Related papers  (2021-05-17T15:40:42Z)
• Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images [58.287120077778205]
  We introduce a framework for analyzing robustness of the learning algorithm w.r.t varying quality in the image input for autonomous driving. Using the results of sensitivity analysis, we propose an algorithm to improve the overall performance of the task of "learning to steer"
  arXiv  Detail & Related papers  (2021-02-26T02:08:07Z)
• Active Hierarchical Imitation and Reinforcement Learning [0.0]
  In this project, we explored different imitation learning algorithms and designed active learning algorithms upon the hierarchical imitation and reinforcement learning framework we have developed. Our experimental results showed that using DAgger and reward-based active learning method can achieve better performance while saving more human efforts physically and mentally during the training process.
  arXiv  Detail & Related papers  (2020-12-14T08:27:27Z)
• Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learning [81.12201426668894]
  We develop efficient reinforcement learning methods that acquire diverse skills without any reward function, and then repurpose these skills for downstream tasks. We show that our proposed algorithm provides substantial improvement in learning efficiency, making reward-free real-world training feasible. We also demonstrate that the learned skills can be composed using model predictive control for goal-oriented navigation, without any additional training.
  arXiv  Detail & Related papers  (2020-04-27T17:38:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.