End-to-End Urban Driving by Imitating a Reinforcement Learning Coach
- URL: http://arxiv.org/abs/2108.08265v1
- Date: Wed, 18 Aug 2021 17:36:51 GMT
- Title: End-to-End Urban Driving by Imitating a Reinforcement Learning Coach
- Authors: Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, Luc Van Gool
- Abstract summary: Humans are good drivers, but not good coaches for end-to-end algorithms.
We train a reinforcement learning expert that maps bird's-eye view images to continuous low-level actions.
Supervised by our reinforcement learning coach, a baseline end-to-end agent with monocular camera-input achieves expert-level performance.
- Score: 148.2683592850329
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: End-to-end approaches to autonomous driving commonly rely on expert
demonstrations. Although humans are good drivers, they are not good coaches for
end-to-end algorithms that demand dense on-policy supervision. On the contrary,
automated experts that leverage privileged information can efficiently generate
large scale on-policy and off-policy demonstrations. However, existing
automated experts for urban driving make heavy use of hand-crafted rules and
perform suboptimally even on driving simulators, where ground-truth information
is available. To address these issues, we train a reinforcement learning expert
that maps bird's-eye view images to continuous low-level actions. While setting
a new performance upper-bound on CARLA, our expert is also a better coach that
provides informative supervision signals for imitation learning agents to learn
from. Supervised by our reinforcement learning coach, a baseline end-to-end
agent with monocular camera-input achieves expert-level performance. Our
end-to-end agent achieves a 78% success rate while generalizing to a new town
and new weather on the NoCrash-dense benchmark and state-of-the-art performance
on the more challenging CARLA LeaderBoard.
Related papers
- GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching [82.19172267487998]
GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching.
This paper introduces GARLIC: a framework of GPT-Augmented Reinforcement Learning with Intelligent Control for vehicle dispatching.
arXiv Detail & Related papers (2024-08-19T08:23:38Z) - Guiding Attention in End-to-End Driving Models [49.762868784033785]
Vision-based end-to-end driving models trained by imitation learning can lead to affordable solutions for autonomous driving.
We study how to guide the attention of these models to improve their driving quality by adding a loss term during training.
In contrast to previous work, our method does not require these salient semantic maps to be available during testing time.
arXiv Detail & Related papers (2024-04-30T23:18:51Z) - Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents.
We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead.
Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
arXiv Detail & Related papers (2023-06-27T17:58:39Z) - Learning energy-efficient driving behaviors by imitating experts [75.12960180185105]
This paper examines the role of imitation learning in bridging the gap between control strategies and realistic limitations in communication and sensing.
We show that imitation learning can succeed in deriving policies that, if adopted by 5% of vehicles, may boost the energy-efficiency of networks with varying traffic conditions by 15% using only local observations.
arXiv Detail & Related papers (2022-06-28T17:08:31Z) - CADRE: A Cascade Deep Reinforcement Learning Framework for Vision-based
Autonomous Urban Driving [43.269130988225605]
Vision-based autonomous urban driving in dense traffic is quite challenging due to the complicated urban environment and the dynamics of the driving behaviors.
We present a novel CAscade Deep REinforcement learning framework, CADRE, to achieve model-free vision-based autonomous urban driving.
arXiv Detail & Related papers (2022-02-17T10:07:16Z) - Urban Driver: Learning to Drive from Real-world Demonstrations Using
Policy Gradients [7.58363583085932]
We present an offline policy method for learning imitative policies for complex urban driving from a large corpus of real-world demonstrations.
This is achieved by building a differentiable data-driven simulator on top of perception outputs and high-fidelity HD maps of the area.
We train our proposed method on 100 hours of expert demonstrations on urban roads and show that it learns complex driving policies that generalize well and can perform a variety of driving maneuvers.
arXiv Detail & Related papers (2021-09-27T20:19:18Z) - Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement
Learning [13.699336307578488]
Deep imitative reinforcement learning approach (DIRL) achieves agile autonomous racing using visual inputs.
We validate our algorithm both in a high-fidelity driving simulation and on a real-world 1/20-scale RC-car with limited onboard computation.
arXiv Detail & Related papers (2021-07-18T00:00:48Z) - Autonomous Overtaking in Gran Turismo Sport Using Curriculum
Reinforcement Learning [39.757652701917166]
This work proposes a new learning-based method to tackle the autonomous overtaking problem.
We evaluate our approach using Gran Turismo Sport -- a world-leading car racing simulator.
arXiv Detail & Related papers (2021-03-26T18:06:50Z) - Affordance-based Reinforcement Learning for Urban Driving [3.507764811554557]
We propose a deep reinforcement learning framework to learn optimal control policy using waypoints and low-dimensional visual representations.
We demonstrate that our agents when trained from scratch learn the tasks of lane-following, driving around inter-sections as well as stopping in front of other actors or traffic lights even in the dense traffic setting.
arXiv Detail & Related papers (2021-01-15T05:21:25Z) - Learning by Cheating [72.9701333689606]
We show that this challenging learning problem can be simplified by decomposing it into two stages.
We use the presented approach to train a vision-based autonomous driving system that substantially outperforms the state of the art.
Our approach achieves, for the first time, 100% success rate on all tasks in the original CARLA benchmark, sets a new record on the NoCrash benchmark, and reduces the frequency of infractions by an order of magnitude compared to the prior state of the art.
arXiv Detail & Related papers (2019-12-27T18:59:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.