Language Conditioned Traffic Generation

• URL: http://arxiv.org/abs/2307.07947v1
• Date: Sun, 16 Jul 2023 05:10:32 GMT
• Title: Language Conditioned Traffic Generation
• Authors: Shuhan Tan, Boris Ivanovic, Xinshuo Weng, Marco Pavone, Philipp Kraehenbuehl
• Abstract summary: LCTGen is a large language model with a transformer-based decoder architecture that selects likely map locations from a dataset of maps. It produces an initial traffic distribution, as well as the dynamics of each vehicle. LCTGen outperforms prior work in both unconditional and conditional traffic scene generation in terms of realism and fidelity.
• Score: 37.71751991840586
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Simulation forms the backbone of modern self-driving development. Simulators help develop, test, and improve driving systems without putting humans, vehicles, or their environment at risk. However, simulators face a major challenge: They rely on realistic, scalable, yet interesting content. While recent advances in rendering and scene reconstruction make great strides in creating static scene assets, modeling their layout, dynamics, and behaviors remains challenging. In this work, we turn to language as a source of supervision for dynamic traffic scene generation. Our model, LCTGen, combines a large language model with a transformer-based decoder architecture that selects likely map locations from a dataset of maps, and produces an initial traffic distribution, as well as the dynamics of each vehicle. LCTGen outperforms prior work in both unconditional and conditional traffic scene generation in terms of realism and fidelity. Code and video will be available at https://ariostgx.github.io/lctgen.

Related papers

• DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Driving [30.024309081789053]
  DriveArena is a high-fidelity closed-loop simulation system designed for driving agents navigating in real scenarios. It features Traffic Manager, a traffic simulator capable of generating realistic traffic flow on any worldwide street map, and World Dreamer, a high-fidelity conditional generative model with infinite autoregression.
  arXiv  Detail & Related papers  (2024-08-01T09:32:01Z)
• SimGen: Simulator-conditioned Driving Scene Generation [50.03358485083602]
  We introduce a simulator-conditioned scene generation framework called SimGen. SimGen learns to generate diverse driving scenes by mixing data from the simulator and the real world. It achieves superior generation quality and diversity while preserving controllability based on the text prompt and the layout pulled from a simulator.
  arXiv  Detail & Related papers  (2024-06-13T17:58:32Z)
• RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios [58.62407014256686]
  RealGen is a novel retrieval-based in-context learning framework for traffic scenario generation. RealGen synthesizes new scenarios by combining behaviors from multiple retrieved examples in a gradient-free way. This in-context learning framework endows versatile generative capabilities, including the ability to edit scenarios.
  arXiv  Detail & Related papers  (2023-12-19T23:11:06Z)
• Model-Based Imitation Learning for Urban Driving [26.782783239210087]
  We present MILE: a Model-based Imitation LEarning approach to jointly learn a model of the world and a policy for autonomous driving. Our model is trained on an offline corpus of urban driving data, without any online interaction with the environment. Our approach is the first camera-only method that models static scene, dynamic scene, and ego-behaviour in an urban driving environment.
  arXiv  Detail & Related papers  (2022-10-14T11:59:46Z)
• A Shared Representation for Photorealistic Driving Simulators [83.5985178314263]
  We propose to improve the quality of generated images by rethinking the discriminator architecture. The focus is on the class of problems where images are generated given semantic inputs, such as scene segmentation maps or human body poses. We aim to learn a shared latent representation that encodes enough information to jointly do semantic segmentation, content reconstruction, along with a coarse-to-fine grained adversarial reasoning.
  arXiv  Detail & Related papers  (2021-12-09T18:59:21Z)
• DriveGAN: Towards a Controllable High-Quality Neural Simulation [147.6822288981004]
  We introduce a novel high-quality neural simulator referred to as DriveGAN. DriveGAN achieves controllability by disentangling different components without supervision. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data.
  arXiv  Detail & Related papers  (2021-04-30T15:30:05Z)
• SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
  We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions. We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
  arXiv  Detail & Related papers  (2021-01-16T22:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.