Learning from Demonstrations of Critical Driving Behaviours Using
Driver's Risk Field
- URL: http://arxiv.org/abs/2210.01747v2
- Date: Sat, 1 Apr 2023 01:38:22 GMT
- Title: Learning from Demonstrations of Critical Driving Behaviours Using
Driver's Risk Field
- Authors: Yurui Du, Flavia Sofia Acerbo, Jens Kober, Tong Duy Son
- Abstract summary: imitation learning (IL) has been widely used in industry as the core of autonomous vehicle (AV) planning modules.
Previous IL works show sample inefficiency and low generalisation in safety-critical scenarios, on which they are rarely tested.
We present an IL model using the spline coefficient parameterisation and offline expert queries to enhance safety and training efficiency.
- Score: 4.272601420525791
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, imitation learning (IL) has been widely used in industry as
the core of autonomous vehicle (AV) planning modules. However, previous IL
works show sample inefficiency and low generalisation in safety-critical
scenarios, on which they are rarely tested. As a result, IL planners can reach
a performance plateau where adding more training data ceases to improve the
learnt policy. First, our work presents an IL model using the spline
coefficient parameterisation and offline expert queries to enhance safety and
training efficiency. Then, we expose the weakness of the learnt IL policy by
synthetically generating critical scenarios through optimisation of parameters
of the driver's risk field (DRF), a parametric human driving behaviour model
implemented in a multi-agent traffic simulator based on the Lyft Prediction
Dataset. To continuously improve the learnt policy, we retrain the IL model
with augmented data. Thanks to the expressivity and interpretability of the
DRF, the desired driving behaviours can be encoded and aggregated to the
original training data. Our work constitutes a full development cycle that can
efficiently and continuously improve the learnt IL policies in closed-loop.
Finally, we show that our IL planner developed with less training resource
still has superior performance compared to the previous state-of-the-art.
Related papers
- MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot
Learning [52.101643259906915]
We study the problem of offline pre-training and online fine-tuning for reinforcement learning from high-dimensional observations.
Existing model-based offline RL methods are not suitable for offline-to-online fine-tuning in high-dimensional domains.
We propose an on-policy model-based method that can efficiently reuse prior data through model-based value expansion and policy regularization.
arXiv Detail & Related papers (2024-01-06T21:04:31Z) - Symbolic Imitation Learning: From Black-Box to Explainable Driving
Policies [5.977871949434069]
We introduce Symbolic Learning (SIL) to learn driving policies which are transparent, explainable and generalisable from available datasets.
Our results demonstrate that SIL not only enhances the interpretability of driving policies but also significantly improves their applicability across varied driving situations.
arXiv Detail & Related papers (2023-09-27T21:03:45Z) - PILOT: A Pre-Trained Model-Based Continual Learning Toolbox [71.63186089279218]
This paper introduces a pre-trained model-based continual learning toolbox known as PILOT.
On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt.
On the other hand, PILOT fits typical class-incremental learning algorithms within the context of pre-trained models to evaluate their effectiveness.
arXiv Detail & Related papers (2023-09-13T17:55:11Z) - When Demonstrations Meet Generative World Models: A Maximum Likelihood
Framework for Offline Inverse Reinforcement Learning [62.00672284480755]
This paper aims to recover the structure of rewards and environment dynamics that underlie observed actions in a fixed, finite set of demonstrations from an expert agent.
Accurate models of expertise in executing a task has applications in safety-sensitive applications such as clinical decision making and autonomous driving.
arXiv Detail & Related papers (2023-02-15T04:14:20Z) - Output Feedback Tube MPC-Guided Data Augmentation for Robust, Efficient
Sensorimotor Policy Learning [49.05174527668836]
Imitation learning (IL) can generate computationally efficient sensorimotor policies from demonstrations provided by computationally expensive model-based sensing and control algorithms.
In this work, we combine IL with an output feedback robust tube model predictive controller to co-generate demonstrations and a data augmentation strategy to efficiently learn neural network-based sensorimotor policies.
We numerically demonstrate that our method can learn a robust visuomotor policy from a single demonstration--a two-orders of magnitude improvement in demonstration efficiency compared to existing IL methods.
arXiv Detail & Related papers (2022-10-18T19:59:17Z) - On the Choice of Data for Efficient Training and Validation of
End-to-End Driving Models [32.381828309166195]
We investigate the influence of several data design choices regarding training and validation of deep driving models trainable in an end-to-end fashion.
We show by correlation analysis, which validation design enables the driving performance measured during validation to generalize to unknown test environments.
arXiv Detail & Related papers (2022-06-01T16:25:28Z) - Model Generation with Provable Coverability for Offline Reinforcement
Learning [14.333861814143718]
offline optimization with dynamics-aware policy provides a new perspective for policy learning and out-of-distribution generalization.
But due to the limitation under the offline setting, the learned model could not mimic real dynamics well enough to support reliable out-of-distribution exploration.
We propose an algorithm to generate models optimizing their coverage for the real dynamics.
arXiv Detail & Related papers (2022-06-01T08:34:09Z) - Learning Interactive Driving Policies via Data-driven Simulation [125.97811179463542]
Data-driven simulators promise high data-efficiency for driving policy learning.
Small underlying datasets often lack interesting and challenging edge cases for learning interactive driving.
We propose a simulation method that uses in-painted ado vehicles for learning robust driving policies.
arXiv Detail & Related papers (2021-11-23T20:14:02Z) - UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning
Leveraging Planning [1.1339580074756188]
Offline reinforcement learning (RL) provides a framework for learning decision-making from offline data.
Self-driving vehicles (SDV) learn a policy, which potentially even outperforms the behavior in the sub-optimal data set.
This motivates the use of model-based offline RL approaches, which leverage planning.
arXiv Detail & Related papers (2021-11-22T10:37:52Z) - Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive
Deep Reinforcement Learning [10.040113551761792]
We use deep reinforcement learning (DRL) to train lidar-based end-to-end driving policies.
In this work, we use DRL to train lidar-based end-to-end driving policies that naturally consider imperfect partial observations.
Our method achieves higher success rates than the state-of-the-art (SOTA) lidar-based end-to-end driving network.
arXiv Detail & Related papers (2021-09-17T11:24:10Z) - Simplifying Deep Reinforcement Learning via Self-Supervision [51.2400839966489]
Self-Supervised Reinforcement Learning (SSRL) is a simple algorithm that optimize policies with purely supervised losses.
We show that SSRL is surprisingly competitive to contemporary algorithms with more stable performance and less running time.
arXiv Detail & Related papers (2021-06-10T06:29:59Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.