DeFIX: Detecting and Fixing Failure Scenarios with Reinforcement Learning in Imitation Learning Based Autonomous Driving

• URL: http://arxiv.org/abs/2210.16567v1
• Date: Sat, 29 Oct 2022 10:58:43 GMT
• Title: DeFIX: Detecting and Fixing Failure Scenarios with Reinforcement Learning in Imitation Learning Based Autonomous Driving
• Authors: Resul Dagdanov, Feyza Eksen, Halil Durmus, Ferhat Yurdakul, Nazim Kemal Ure
• Abstract summary: We present a Reinforcement Learning (RL) based methodology to DEtect and FIX failures of an IL agent. DeFIX is a continuous learning framework, where extraction of failure scenarios and training of RL agents are executed in an infinite loop. It is demonstrated that even with only one RL agent trained on failure scenario of an IL agent, DeFIX method is either competitive or does outperform state-of-the-art IL and RL based autonomous urban driving benchmarks.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Safely navigating through an urban environment without violating any traffic rules is a crucial performance target for reliable autonomous driving. In this paper, we present a Reinforcement Learning (RL) based methodology to DEtect and FIX (DeFIX) failures of an Imitation Learning (IL) agent by extracting infraction spots and re-constructing mini-scenarios on these infraction areas to train an RL agent for fixing the shortcomings of the IL approach. DeFIX is a continuous learning framework, where extraction of failure scenarios and training of RL agents are executed in an infinite loop. After each new policy is trained and added to the library of policies, a policy classifier method effectively decides on which policy to activate at each step during the evaluation. It is demonstrated that even with only one RL agent trained on failure scenario of an IL agent, DeFIX method is either competitive or does outperform state-of-the-art IL and RL based autonomous urban driving benchmarks. We trained and validated our approach on the most challenging map (Town05) of CARLA simulator which involves complex, realistic, and adversarial driving scenarios. The source code is publicly available at https://github.com/data-and-decision-lab/DeFIX

Related papers

• CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving [45.05135725542318]
  We propose a framework that enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. By combining RL and imitation, we demonstrate that our method achieves state-of-the-art results in closed loop simulation driving benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T07:31:29Z)
• Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
  We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents. We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead. Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
  arXiv  Detail & Related papers  (2023-06-27T17:58:39Z)
• A Multiplicative Value Function for Safe and Efficient Reinforcement Learning [131.96501469927733]
  We propose a safe model-free RL algorithm with a novel multiplicative value function consisting of a safety critic and a reward critic. The safety critic predicts the probability of constraint violation and discounts the reward critic that only estimates constraint-free returns. We evaluate our method in four safety-focused environments, including classical RL benchmarks augmented with safety constraints and robot navigation tasks with images and raw Lidar scans as observations.
  arXiv  Detail & Related papers  (2023-03-07T18:29:15Z)
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning [64.11013095004786]
  We propose a dual-agent safe reinforcement learning strategy consisting of a baseline and a safe agent. Such a decoupled framework enables high flexibility, data efficiency and risk-awareness for RL-based control. The proposed method outperforms the state-of-the-art safe RL algorithms on difficult robot locomotion and manipulation tasks.
  arXiv  Detail & Related papers  (2022-12-14T03:11:25Z)
• Self-Improving Safety Performance of Reinforcement Learning Based Driving with Black-Box Verification Algorithms [0.0]
  We propose a self-improving artificial intelligence system to enhance the safety performance of reinforcement learning (RL)-based autonomous driving (AD) agents. Our approach efficiently discovers safety failures of action decisions in RL-based adaptive cruise control (ACC) applications.
  arXiv  Detail & Related papers  (2022-10-29T11:34:17Z)
• FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge Computing Migrations [52.85536740465277]
  FIRE is a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment. We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function. We show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.
  arXiv  Detail & Related papers  (2022-09-28T19:49:39Z)
• DriverGym: Democratising Reinforcement Learning for Autonomous Driving [75.91049219123899]
  We propose DriverGym, an open-source environment for developing reinforcement learning algorithms for autonomous driving. DriverGym provides access to more than 1000 hours of expert logged data and also supports reactive and data-driven agent behavior. The performance of an RL policy can be easily validated on real-world data using our extensive and flexible closed-loop evaluation protocol.
  arXiv  Detail & Related papers  (2021-11-12T11:47:08Z)
• A Reinforcement Learning Benchmark for Autonomous Driving in Intersection Scenarios [11.365750371241154]
  We propose a benchmark for training and testing RL-based autonomous driving agents in complex intersection scenarios, which is called RL-CIS. The test benchmark and baselines are to provide a fair and comprehensive training and testing platform for the study of RL for autonomous driving in the intersection scenario.
  arXiv  Detail & Related papers  (2021-09-22T07:38:23Z)
• Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning [13.699336307578488]
  Deep imitative reinforcement learning approach (DIRL) achieves agile autonomous racing using visual inputs. We validate our algorithm both in a high-fidelity driving simulation and on a real-world 1/20-scale RC-car with limited onboard computation.
  arXiv  Detail & Related papers  (2021-07-18T00:00:48Z)
• CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous driving [8.287331387095545]
  This work introduces interactive traffic scenarios in the CARLA simulator, which are based on real-world traffic. We concentrate on tactical tasks lasting several seconds, which are especially challenging for current control methods. The CARLA Real Traffic Scenarios (CRTS) is intended to be a training and testing ground for autonomous driving systems.
  arXiv  Detail & Related papers  (2020-12-16T13:20:39Z)
• Learning from Simulation, Racing in Reality [126.56346065780895]
  We present a reinforcement learning-based solution to autonomously race on a miniature race car platform. We show that a policy that is trained purely in simulation can be successfully transferred to the real robotic setup.
  arXiv  Detail & Related papers  (2020-11-26T14:58:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.