TrafficBots: Towards World Models for Autonomous Driving Simulation and
Motion Prediction
- URL: http://arxiv.org/abs/2303.04116v2
- Date: Thu, 28 Sep 2023 14:50:40 GMT
- Title: TrafficBots: Towards World Models for Autonomous Driving Simulation and
Motion Prediction
- Authors: Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, Luc Van Gool
- Abstract summary: We show data-driven traffic simulation can be formulated as a world model.
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving.
Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
- Score: 149.5716746789134
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Data-driven simulation has become a favorable way to train and test
autonomous driving algorithms. The idea of replacing the actual environment
with a learned simulator has also been explored in model-based reinforcement
learning in the context of world models. In this work, we show data-driven
traffic simulation can be formulated as a world model. We present TrafficBots,
a multi-agent policy built upon motion prediction and end-to-end driving, and
based on TrafficBots we obtain a world model tailored for the planning module
of autonomous vehicles. Existing data-driven traffic simulators are lacking
configurability and scalability. To generate configurable behaviors, for each
agent we introduce a destination as navigational information, and a
time-invariant latent personality that specifies the behavioral style. To
improve the scalability, we present a new scheme of positional encoding for
angles, allowing all agents to share the same vectorized context and the use of
an architecture based on dot-product attention. As a result, we can simulate
all traffic participants seen in dense urban scenarios. Experiments on the
Waymo open motion dataset show TrafficBots can simulate realistic multi-agent
behaviors and achieve good performance on the motion prediction task.
Related papers
- Solving Motion Planning Tasks with a Scalable Generative Model [15.858076912795621]
We present an efficient solution based on generative models which learns the dynamics of the driving scenes.
Our innovative design allows the model to operate in both full-Autoregressive and partial-Autoregressive modes.
We conclude that the proposed generative model may serve as a foundation for a variety of motion planning tasks.
arXiv Detail & Related papers (2024-07-03T03:57:05Z) - Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs.
We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios.
Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
arXiv Detail & Related papers (2023-12-07T18:53:27Z) - Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous
Driving Research [76.93956925360638]
Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes.
It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training.
We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
arXiv Detail & Related papers (2023-10-12T20:49:15Z) - BITS: Bi-level Imitation for Traffic Simulation [38.28736985320897]
We take a data-driven approach and propose a method that can learn to generate traffic behaviors from real-world driving logs.
We empirically validate our method, named Bi-level Imitation for Traffic Simulation (BITS), with scenarios from two large-scale driving datasets.
As part of our core contributions, we develop and open source a software tool that unifies data formats across different driving datasets.
arXiv Detail & Related papers (2022-08-26T02:17:54Z) - A Hierarchical Pedestrian Behavior Model to Generate Realistic Human
Behavior in Traffic Simulation [11.525073205608681]
We present a hierarchical pedestrian behavior model that generates high-level decisions through the use of behavior trees.
A full implementation of our work is integrated into GeoScenario Server, a scenario definition and execution engine.
Our model is shown to replicate the real-world pedestrians' trajectories with a high degree of fidelity and a decision-making accuracy of 98% or better.
arXiv Detail & Related papers (2022-06-01T02:04:38Z) - VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and
Policy Learning for Autonomous Vehicles [131.2240621036954]
We present VISTA, an open source, data-driven simulator that integrates multiple types of sensors for autonomous vehicles.
Using high fidelity, real-world datasets, VISTA represents and simulates RGB cameras, 3D LiDAR, and event-based cameras.
We demonstrate the ability to train and test perception-to-control policies across each of the sensor types and showcase the power of this approach via deployment on a full scale autonomous vehicle.
arXiv Detail & Related papers (2021-11-23T18:58:10Z) - Towards Optimal Strategies for Training Self-Driving Perception Models
in Simulation [98.51313127382937]
We focus on the use of labels in the synthetic domain alone.
Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator.
We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
arXiv Detail & Related papers (2021-11-15T18:37:43Z) - TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation.
In particular, we leverage an implicit latent variable model to parameterize a joint actor policy.
We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
arXiv Detail & Related papers (2021-01-17T00:29:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.