HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving

• URL: http://arxiv.org/abs/2401.03160v5
• Date: Fri, 14 Jun 2024 23:00:31 GMT
• Title: HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving
• Authors: Zilin Huang, Zihao Sheng, Chengyuan Ma, Sikai Chen,
• Abstract summary: We propose an enhanced human-in-the-loop reinforcement learning method, termed the Human as AI mentor-based deep reinforcement learning (HAIM-DRL) framework. We first introduce an innovative learning paradigm that effectively injects human intelligence into AI, termed Human as AI mentor (HAIM) In this paradigm, the human expert serves as a mentor to the AI agent, while the agent could be guided to minimize traffic flow disturbance.
• Score: 2.807187711407621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite significant progress in autonomous vehicles (AVs), the development of driving policies that ensure both the safety of AVs and traffic flow efficiency has not yet been fully explored. In this paper, we propose an enhanced human-in-the-loop reinforcement learning method, termed the Human as AI mentor-based deep reinforcement learning (HAIM-DRL) framework, which facilitates safe and efficient autonomous driving in mixed traffic platoon. Drawing inspiration from the human learning process, we first introduce an innovative learning paradigm that effectively injects human intelligence into AI, termed Human as AI mentor (HAIM). In this paradigm, the human expert serves as a mentor to the AI agent. While allowing the agent to sufficiently explore uncertain environments, the human expert can take control in dangerous situations and demonstrate correct actions to avoid potential accidents. On the other hand, the agent could be guided to minimize traffic flow disturbance, thereby optimizing traffic flow efficiency. In detail, HAIM-DRL leverages data collected from free exploration and partial human demonstrations as its two training sources. Remarkably, we circumvent the intricate process of manually designing reward functions; instead, we directly derive proxy state-action values from partial human demonstrations to guide the agents' policy learning. Additionally, we employ a minimal intervention technique to reduce the human mentor's cognitive load. Comparative results show that HAIM-DRL outperforms traditional methods in driving safety, sampling efficiency, mitigation of traffic flow disturbance, and generalizability to unseen traffic scenarios. The code and demo videos for this paper can be accessed at: https://zilin-huang.github.io/HAIM-DRL-website/

Related papers

• Ego-Foresight: Agent Visuomotor Prediction as Regularization for RL [34.6883445484835]
  Ego-Foresight is a self-supervised method for disentangling agent and environment based on motion and prediction. We show that visuomotor prediction of the agent provides regularization to the RL algorithm, by encouraging the actions to stay within predictable bounds. We integrate Ego-Foresight with a model-free RL algorithm to solve simulated robotic manipulation tasks, showing an average improvement of 23% in efficiency and 8% in performance.
  arXiv  Detail & Related papers  (2024-05-27T13:32:43Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• Multi-Agent Deep Reinforcement Learning for Dynamic Avatar Migration in AIoT-enabled Vehicular Metaverses with Trajectory Prediction [70.9337170201739]
  We propose a model to predict the future trajectories of intelligent vehicles based on their historical data. We show that our proposed algorithm can effectively reduce the latency of executing avatar tasks by around 25% without prediction.
  arXiv  Detail & Related papers  (2023-06-26T13:27:11Z)
• FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing [71.76084256567599]
  We present a system that enables an autonomous small-scale RC car to drive aggressively from visual observations using reinforcement learning (RL) Our system, FastRLAP (faster lap), trains autonomously in the real world, without human interventions, and without requiring any simulation or expert demonstrations. The resulting policies exhibit emergent aggressive driving skills, such as timing braking and acceleration around turns and avoiding areas which impede the robot's motion, approaching the performance of a human driver using a similar first-person interface over the course of training.
  arXiv  Detail & Related papers  (2023-04-19T17:33:47Z)
• Constrained Reinforcement Learning for Robotics via Scenario-Based Programming [64.07167316957533]
  It is crucial to optimize the performance of DRL-based agents while providing guarantees about their behavior. This paper presents a novel technique for incorporating domain-expert knowledge into a constrained DRL training loop. Our experiments demonstrate that using our approach to leverage expert knowledge dramatically improves the safety and the performance of the agent.
  arXiv  Detail & Related papers  (2022-06-20T07:19:38Z)
• Human-AI Shared Control via Frequency-based Policy Dissection [34.0399894373716]
  Human-AI shared control allows human to interact and collaborate with AI to accomplish control tasks in complex environments. Previous Reinforcement Learning (RL) methods attempt the goal-conditioned design to achieve human-controllable policies. We develop a simple yet effective frequency-based approach called textitPolicy Dissection to align the intermediate representation of the learned neural controller with the kinematic attributes of the agent behavior.
  arXiv  Detail & Related papers  (2022-05-31T23:57:55Z)
• Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimization [38.21629972247463]
  We develop a novel human-in-the-loop learning method called Human-AI Copilot Optimization (HACO) The proposed HACO effectively utilizes the data both from the trial-and-error exploration and human's partial demonstration to train a high-performing agent. experiments show that HACO achieves a substantially high sample efficiency in the safe driving benchmark.
  arXiv  Detail & Related papers  (2022-02-17T06:29:46Z)
• DDPG car-following model with real-world human driving experience in CARLA [0.0]
  We propose a two-stage Deep Reinforcement Learning (DRL) method, that learns from real-world human driving to achieve performance that is superior to the pure DRL agent. For evaluation, we designed different real-world driving scenarios to compare the proposed two-stage DRL agent with the pure DRL agent.
  arXiv  Detail & Related papers  (2021-12-29T15:22:31Z)
• PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training [94.87393610927812]
  We present an off-policy, interactive reinforcement learning algorithm that capitalizes on the strengths of both feedback and off-policy learning. We demonstrate that our approach is capable of learning tasks of higher complexity than previously considered by human-in-the-loop methods.
  arXiv  Detail & Related papers  (2021-06-09T14:10:50Z)
• Accelerating Reinforcement Learning Agent with EEG-based Implicit Human Feedback [10.138798960466222]
  Reinforcement Learning (RL) agents with human feedback can dramatically improve various aspects of learning. Previous methods require human observer to give inputs explicitly, burdening the human in the loop of RL agent's learning process. We investigate capturing human's intrinsic reactions as implicit (and natural) feedback through EEG in the form of error-related potentials (ErrP)
  arXiv  Detail & Related papers  (2020-06-30T03:13:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.