Embedding Synthetic Off-Policy Experience for Autonomous Driving via
Zero-Shot Curricula
- URL: http://arxiv.org/abs/2212.01375v1
- Date: Fri, 2 Dec 2022 18:57:21 GMT
- Title: Embedding Synthetic Off-Policy Experience for Autonomous Driving via
Zero-Shot Curricula
- Authors: Eli Bronstein, Sirish Srinivasan, Supratik Paul, Aman Sinha, Matthew
O'Kelly, Payam Nikdel, Shimon Whiteson
- Abstract summary: We show that an agent trained using only a 10% subset of the data performs just as well as an agent trained on the entire dataset.
We then demonstrate that this difficulty score can be used in a zero-shot transfer to generate curricula for an imitation-learning based planning agent.
- Score: 48.58973705935691
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: ML-based motion planning is a promising approach to produce agents that
exhibit complex behaviors, and automatically adapt to novel environments. In
the context of autonomous driving, it is common to treat all available training
data equally. However, this approach produces agents that do not perform
robustly in safety-critical settings, an issue that cannot be addressed by
simply adding more data to the training set - we show that an agent trained
using only a 10% subset of the data performs just as well as an agent trained
on the entire dataset. We present a method to predict the inherent difficulty
of a driving situation given data collected from a fleet of autonomous vehicles
deployed on public roads. We then demonstrate that this difficulty score can be
used in a zero-shot transfer to generate curricula for an imitation-learning
based planning agent. Compared to training on the entire unbiased training
dataset, we show that prioritizing difficult driving scenarios both reduces
collisions by 15% and increases route adherence by 14% in closed-loop
evaluation, all while using only 10% of the training data.
Related papers
- Perception Without Vision for Trajectory Prediction: Ego Vehicle Dynamics as Scene Representation for Efficient Active Learning in Autonomous Driving [0.0]
We propose methods for clustering trajectory-states and sampling strategies in an active learning framework.
By integrating trajectory-state-informed active learning, we demonstrate that more efficient and robust autonomous driving systems are possible.
arXiv Detail & Related papers (2024-05-15T02:54:11Z) - Rethinking Closed-loop Training for Autonomous Driving [82.61418945804544]
We present the first empirical study which analyzes the effects of different training benchmark designs on the success of learning agents.
We propose trajectory value learning (TRAVL), an RL-based driving agent that performs planning with multistep look-ahead.
Our experiments show that TRAVL can learn much faster and produce safer maneuvers compared to all the baselines.
arXiv Detail & Related papers (2023-06-27T17:58:39Z) - Imitation Is Not Enough: Robustifying Imitation with Reinforcement
Learning for Challenging Driving Scenarios [147.16925581385576]
We show how imitation learning combined with reinforcement learning can substantially improve the safety and reliability of driving policies.
We train a policy on over 100k miles of urban driving data, and measure its effectiveness in test scenarios grouped by different levels of collision likelihood.
arXiv Detail & Related papers (2022-12-21T23:59:33Z) - Safe Real-World Autonomous Driving by Learning to Predict and Plan with
a Mixture of Experts [3.2230833657560503]
We propose a distribution over multiple future trajectories for both the self-driving vehicle and other road agents.
During inference, we select the planning trajectory that minimizes a cost taking into account safety and the predicted probabilities.
We successfully deploy it on a self-driving vehicle on urban public roads, confirming that it drives safely without compromising comfort.
arXiv Detail & Related papers (2022-11-03T20:16:24Z) - Tackling Real-World Autonomous Driving using Deep Reinforcement Learning [63.3756530844707]
In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts acceleration and steering angle.
In order to deploy the system on board the real self-driving car, we also develop a module represented by a tiny neural network.
arXiv Detail & Related papers (2022-07-05T16:33:20Z) - Optimizing Trajectories for Highway Driving with Offline Reinforcement
Learning [11.970409518725491]
We propose a Reinforcement Learning-based approach to autonomous driving.
We compare the performance of our agent against four other highway driving agents.
We demonstrate that our offline trained agent, with randomly collected data, learns to drive smoothly, achieving as close as possible to the desired velocity, while outperforming the other agents.
arXiv Detail & Related papers (2022-03-21T13:13:08Z) - Learning Interactive Driving Policies via Data-driven Simulation [125.97811179463542]
Data-driven simulators promise high data-efficiency for driving policy learning.
Small underlying datasets often lack interesting and challenging edge cases for learning interactive driving.
We propose a simulation method that uses in-painted ado vehicles for learning robust driving policies.
arXiv Detail & Related papers (2021-11-23T20:14:02Z) - Injecting Knowledge in Data-driven Vehicle Trajectory Predictors [82.91398970736391]
Vehicle trajectory prediction tasks have been commonly tackled from two perspectives: knowledge-driven or data-driven.
In this paper, we propose to learn a "Realistic Residual Block" (RRB) which effectively connects these two perspectives.
Our proposed method outputs realistic predictions by confining the residual range and taking into account its uncertainty.
arXiv Detail & Related papers (2021-03-08T16:03:09Z) - Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle
Network [1.599072005190786]
We formulate the problem of passenger-vehicle matching in a sparsely connected graph.
We propose an algorithm to derive an equilibrium policy in a multi-agent environment.
arXiv Detail & Related papers (2021-02-13T03:18:44Z) - Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous
Vehicles [11.180588185127892]
Supervised learning algorithms can generalize to new environments by training on a large amount of labeled data.
It can be often impractical or cost-prohibitive to obtain sufficient data for each new environment.
We propose a meta reinforcement learning (MRL) method to improve the agent's generalization capabilities.
arXiv Detail & Related papers (2020-08-28T02:57:11Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.