SimNet: Learning Reactive Self-driving Simulations from Real-world
Observations
- URL: http://arxiv.org/abs/2105.12332v1
- Date: Wed, 26 May 2021 05:14:23 GMT
- Title: SimNet: Learning Reactive Self-driving Simulations from Real-world
Observations
- Authors: Luca Bergamini, Yawei Ye, Oliver Scheel, Long Chen, Chih Hu, Luca Del
Pero, Blazej Osinski, Hugo Grimmett, Peter Ondruska
- Abstract summary: We present an end-to-end trainable machine learning system capable of realistically simulating driving experiences.
This can be used for the verification of self-driving system performance without relying on expensive and time-consuming road testing.
- Score: 10.035169936164504
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In this work, we present a simple end-to-end trainable machine learning
system capable of realistically simulating driving experiences. This can be
used for the verification of self-driving system performance without relying on
expensive and time-consuming road testing. In particular, we frame the
simulation problem as a Markov Process, leveraging deep neural networks to
model both state distribution and transition function. These are trainable
directly from the existing raw observations without the need for any
handcrafting in the form of plant or kinematic models. All that is needed is a
dataset of historical traffic episodes. Our formulation allows the system to
construct never seen scenes that unfold realistically reacting to the
self-driving car's behaviour. We train our system directly from 1,000 hours of
driving logs and measure both realism, reactivity of the simulation as the two
key properties of the simulation. At the same time, we apply the method to
evaluate the performance of a recently proposed state-of-the-art ML planning
system trained from human driving logs. We discover this planning system is
prone to previously unreported causal confusion issues that are difficult to
test by non-reactive simulation. To the best of our knowledge, this is the
first work that directly merges highly realistic data-driven simulations with a
closed-loop evaluation for self-driving vehicles. We make the data, code, and
pre-trained models publicly available to further stimulate simulation
development.
Related papers
- Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous
Driving Research [76.93956925360638]
Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes.
It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training.
We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
arXiv Detail & Related papers (2023-10-12T20:49:15Z) - Perception Imitation: Towards Synthesis-free Simulator for Autonomous
Vehicles [45.27200446670184]
We propose a perception imitation method to simulate results of a certain perception model, and discuss a new route of autonomous driving simulator without data synthesis.
Experiments show that our method is effective to model the behavior of learning-based perception model, and can be further applied in the proposed simulation route smoothly.
arXiv Detail & Related papers (2023-04-19T01:27:02Z) - TrafficBots: Towards World Models for Autonomous Driving Simulation and
Motion Prediction [149.5716746789134]
We show data-driven traffic simulation can be formulated as a world model.
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving.
Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
arXiv Detail & Related papers (2023-03-07T18:28:41Z) - Autonomous Driving Simulator based on Neurorobotics Platform [11.25880077022107]
There are many artificial intelligence algorithms for autonomous driving, but directly installing these algorithms on vehicles is unrealistic and expensive.
At the same time, many of these algorithms need an environment to train and optimize.
This report will start with a little research on the Neurorobotics Platform and analyze the potential and possibility of developing a new simulator to achieve the true real-world simulation goal.
arXiv Detail & Related papers (2022-12-31T01:12:27Z) - Continual learning autoencoder training for a particle-in-cell
simulation via streaming [52.77024349608834]
upcoming exascale era will provide a new generation of physics simulations with high resolution.
These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible.
This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
arXiv Detail & Related papers (2022-11-09T09:55:14Z) - Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving
Without Real Data [56.49494318285391]
We present Sim2Seg, a re-imagining of RCAN that crosses the visual reality gap for off-road autonomous driving.
This is done by learning to translate randomized simulation images into simulated segmentation and depth maps.
This allows us to train an end-to-end RL policy in simulation, and directly deploy in the real-world.
arXiv Detail & Related papers (2022-10-25T17:50:36Z) - BITS: Bi-level Imitation for Traffic Simulation [38.28736985320897]
We take a data-driven approach and propose a method that can learn to generate traffic behaviors from real-world driving logs.
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets.
As part of our core contributions, we develop and open source a software tool that unifies data formats across different driving datasets.
arXiv Detail & Related papers (2022-08-26T02:17:54Z) - DriveGAN: Towards a Controllable High-Quality Neural Simulation [147.6822288981004]
We introduce a novel high-quality neural simulator referred to as DriveGAN.
DriveGAN achieves controllability by disentangling different components without supervision.
We train DriveGAN on multiple datasets, including 160 hours of real-world driving data.
arXiv Detail & Related papers (2021-04-30T15:30:05Z) - Testing the Safety of Self-driving Vehicles by Simulating Perception and
Prediction [88.0416857308144]
We propose an alternative to sensor simulation, as sensor simulation is expensive and has large domain gaps.
We directly simulate the outputs of the self-driving vehicle's perception and prediction system, enabling realistic motion planning testing.
arXiv Detail & Related papers (2020-08-13T17:20:02Z) - A machine learning environment for evaluating autonomous driving
software [1.6516902135723865]
We present a machine learning environment for detecting autonomous vehicle corner case behavior.
Our environment is based on connecting the CARLA simulation software to machine learning framework and custom AI client software.
Our system can search for corner cases where the vehicle AI is unable to correctly understand the situation.
arXiv Detail & Related papers (2020-03-07T13:05:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.