Language-Guided Traffic Simulation via Scene-Level Diffusion
- URL: http://arxiv.org/abs/2306.06344v2
- Date: Wed, 18 Oct 2023 23:51:14 GMT
- Title: Language-Guided Traffic Simulation via Scene-Level Diffusion
- Authors: Ziyuan Zhong, Davis Rempe, Yuxiao Chen, Boris Ivanovic, Yulong Cao,
Danfei Xu, Marco Pavone, Baishakhi Ray
- Abstract summary: We present CTG++, a scene-level conditional diffusion model that can be guided by language instructions.
We first propose a scene-level diffusion model equipped with atemporal backbone which generates realistic and controllable traffic.
We then harness a large language model (LLM) to convert a users query into a loss function guiding the diffusion model towards query-compliant generation.
- Score: 46.47977644226296
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Realistic and controllable traffic simulation is a core capability that is
necessary to accelerate autonomous vehicle (AV) development. However, current
approaches for controlling learning-based traffic models require significant
domain expertise and are difficult for practitioners to use. To remedy this, we
present CTG++, a scene-level conditional diffusion model that can be guided by
language instructions. Developing this requires tackling two challenges: the
need for a realistic and controllable traffic model backbone, and an effective
method to interface with a traffic model using language. To address these
challenges, we first propose a scene-level diffusion model equipped with a
spatio-temporal transformer backbone, which generates realistic and
controllable traffic. We then harness a large language model (LLM) to convert a
user's query into a loss function, guiding the diffusion model towards
query-compliant generation. Through comprehensive evaluation, we demonstrate
the effectiveness of our proposed method in generating realistic,
query-compliant traffic simulations.
Related papers
- An Approach to Systematic Data Acquisition and Data-Driven Simulation for the Safety Testing of Automated Driving Functions [32.37902846268263]
In R&D areas related to the safety impact of the "open world", there is a significant shortage of real-world data to parameterize and/or validate simulations.
We present an approach to systematically acquire data in public traffic by heterogeneous means, transform it into a unified representation, and use it to automatically parameterize traffic behavior models for use in data-driven virtual validation of automated driving functions.
arXiv Detail & Related papers (2024-05-02T23:24:27Z) - SAFE-SIM: Safety-Critical Closed-Loop Traffic Simulation with Diffusion-Controllable Adversaries [94.84458417662407]
We introduce SAFE-SIM, a controllable closed-loop safety-critical simulation framework.
Our approach yields two distinct advantages: 1) generating realistic long-tail safety-critical scenarios that closely reflect real-world conditions, and 2) providing controllable adversarial behavior for more comprehensive and interactive evaluations.
We validate our framework empirically using the nuScenes and nuPlan datasets across multiple planners, demonstrating improvements in both realism and controllability.
arXiv Detail & Related papers (2023-12-31T04:14:43Z) - Reinforcement Learning with Human Feedback for Realistic Traffic
Simulation [53.85002640149283]
Key element of effective simulation is the incorporation of realistic traffic models that align with human knowledge.
This study identifies two main challenges: capturing the nuances of human preferences on realism and the unification of diverse traffic simulation models.
arXiv Detail & Related papers (2023-09-01T19:29:53Z) - TransWorldNG: Traffic Simulation via Foundation Model [23.16553424318004]
We present TransWordNG, a traffic simulator that uses Data-driven algorithms and Graph Computing techniques to learn traffic dynamics from real data.
The results demonstrate that TransWorldNG can generate more realistic traffic patterns compared to traditional simulators.
arXiv Detail & Related papers (2023-05-25T05:49:30Z) - TrafficBots: Towards World Models for Autonomous Driving Simulation and
Motion Prediction [149.5716746789134]
We show data-driven traffic simulation can be formulated as a world model.
We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving.
Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
arXiv Detail & Related papers (2023-03-07T18:28:41Z) - Guided Conditional Diffusion for Controllable Traffic Simulation [42.198185904248994]
Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles.
Data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic.
We develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time.
arXiv Detail & Related papers (2022-10-31T14:44:59Z) - A Deep Reinforcement Learning Approach for Traffic Signal Control
Optimization [14.455497228170646]
Inefficient traffic signal control methods may cause numerous problems, such as traffic congestion and waste of energy.
This paper first proposes a multi-agent deep deterministic policy gradient (MADDPG) method by extending the actor-critic policy gradient algorithms.
arXiv Detail & Related papers (2021-07-13T14:11:04Z) - Multi-intersection Traffic Optimisation: A Benchmark Dataset and a
Strong Baseline [85.9210953301628]
Control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas.
Because of the high complexity of modelling the problem, experimental settings of current works are often inconsistent.
We propose a novel and strong baseline model based on deep reinforcement learning with the encoder-decoder structure.
arXiv Detail & Related papers (2021-01-24T03:55:39Z) - TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation.
In particular, we leverage an implicit latent variable model to parameterize a joint actor policy.
We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
arXiv Detail & Related papers (2021-01-17T00:29:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.