Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios

• URL: http://arxiv.org/abs/2212.11419v2
• Date: Thu, 10 Aug 2023 19:29:11 GMT
• Title: Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios
• Authors: Yiren Lu, Justin Fu, George Tucker, Xinlei Pan, Eli Bronstein, Rebecca Roelofs, Benjamin Sapp, Brandyn White, Aleksandra Faust, Shimon Whiteson, Dragomir Anguelov, Sergey Levine
• Abstract summary: We show how imitation learning combined with reinforcement learning can substantially improve the safety and reliability of driving policies. We train a policy on over 100k miles of urban driving data, and measure its effectiveness in test scenarios grouped by different levels of collision likelihood.
• Score: 147.16925581385576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Imitation learning (IL) is a simple and powerful way to use high-quality human driving data, which can be collected at scale, to produce human-like behavior. However, policies based on imitation learning alone often fail to sufficiently account for safety and reliability concerns. In this paper, we show how imitation learning combined with reinforcement learning using simple rewards can substantially improve the safety and reliability of driving policies over those learned from imitation alone. In particular, we train a policy on over 100k miles of urban driving data, and measure its effectiveness in test scenarios grouped by different levels of collision likelihood. Our analysis shows that while imitation can perform well in low-difficulty scenarios that are well-covered by the demonstration data, our proposed approach significantly improves robustness on the most challenging scenarios (over 38% reduction in failures). To our knowledge, this is the first application of a combined imitation and reinforcement learning approach in autonomous driving that utilizes large amounts of real-world human driving data.

Related papers

• IGDrivSim: A Benchmark for the Imitation Gap in Autonomous Driving [35.64960921334498]
  textbfIGDrivSim is a benchmark built on top of the Waymax simulator. Our experiments show that this perception gap can hinder the learning of safe and effective driving behaviors. We show that combining imitation with reinforcement learning, using a simple penalty reward for prohibited behaviors, effectively mitigates these failures.
  arXiv  Detail & Related papers  (2024-11-07T12:28:52Z)
• CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Driving [45.05135725542318]
  We propose a framework that enables training driving policies in simulation through leveraging imitative motion priors and safety constraints. By combining RL and imitation, we demonstrate that our method achieves state-of-the-art results in closed loop simulation driving benchmarks.
  arXiv  Detail & Related papers  (2024-06-13T07:31:29Z)
• Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers? [60.51287814584477]
  This paper evaluates the inherent risks in autonomous driving by examining the current landscape of AVs. We develop specific claims highlighting the delicate balance between the advantages of AVs and potential security challenges in real-world scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:42:21Z)
• RACER: Epistemic Risk-Sensitive RL Enables Fast Driving with Fewer Crashes [57.319845580050924]
  We propose a reinforcement learning framework that combines risk-sensitive control with an adaptive action space curriculum. We show that our algorithm is capable of learning high-speed policies for a real-world off-road driving task.
  arXiv  Detail & Related papers  (2024-05-07T23:32:36Z)
• Evaluation of Safety Constraints in Autonomous Navigation with Deep Reinforcement Learning [62.997667081978825]
  We compare two learnable navigation policies: safe and unsafe. The safe policy takes the constraints into the account, while the other does not. We show that the safe policy is able to generate trajectories with more clearance (distance to the obstacles) and makes less collisions while training without sacrificing the overall performance.
  arXiv  Detail & Related papers  (2023-07-27T01:04:57Z)
• Learning Interactive Driving Policies via Data-driven Simulation [125.97811179463542]
  Data-driven simulators promise high data-efficiency for driving policy learning. Small underlying datasets often lack interesting and challenging edge cases for learning interactive driving. We propose a simulation method that uses in-painted ado vehicles for learning robust driving policies.
  arXiv  Detail & Related papers  (2021-11-23T20:14:02Z)
• Action-Based Representation Learning for Autonomous Driving [8.296684637620551]
  We propose to use action-based driving data for learning representations. Our experiments show that an affordance-based driving model pre-trained with this approach can leverage a relatively small amount of weakly annotated imagery.
  arXiv  Detail & Related papers  (2020-08-21T10:49:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.