DragTraffic: Interactive and Controllable Traffic Scene Generation for Autonomous Driving

• URL: http://arxiv.org/abs/2404.12624v2
• Date: Thu, 10 Oct 2024 14:51:28 GMT
• Title: DragTraffic: Interactive and Controllable Traffic Scene Generation for Autonomous Driving
• Authors: Sheng Wang, Ge Sun, Fulong Ma, Tianshuai Hu, Qiang Qin, Yongkang Song, Lei Zhu, Junwei Liang,
• Abstract summary: DragTraffic is a general, interactive, and controllable traffic scene generation framework based on conditional diffusion. We employ a regression model to provide a general initial solution and a refinement process based on the conditional diffusion model to ensure diversity. Experiments on a real-world driving dataset show that DragTraffic outperforms existing methods in terms of authenticity, diversity, and freedom.
• Score: 10.90477019946728
• License:
• Abstract: Evaluating and training autonomous driving systems require diverse and scalable corner cases. However, most existing scene generation methods lack controllability, accuracy, and versatility, resulting in unsatisfactory generation results. Inspired by DragGAN in image generation, we propose DragTraffic, a generalized, interactive, and controllable traffic scene generation framework based on conditional diffusion. DragTraffic enables non-experts to generate a variety of realistic driving scenarios for different types of traffic agents through an adaptive mixture expert architecture. We employ a regression model to provide a general initial solution and a refinement process based on the conditional diffusion model to ensure diversity. User-customized context is introduced through cross-attention to ensure high controllability. Experiments on a real-world driving dataset show that DragTraffic outperforms existing methods in terms of authenticity, diversity, and freedom. Demo videos and code are available at https://chantsss.github.io/Dragtraffic/.

Related papers

• MagicDriveDiT: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control [68.74166535159311]
  We introduce MagicDriveDiT, a novel approach based on the DiT architecture. By incorporating spatial-temporal conditional encoding, MagicDriveDiT achieves precise control over spatial-temporal latents. Experiments show its superior performance in generating realistic street scene videos with higher resolution and more frames.
  arXiv  Detail & Related papers  (2024-11-21T03:13:30Z)
• WcDT: World-centric Diffusion Transformer for Traffic Scene Generation [13.616763172038846]
  We introduce a novel approach for autonomous driving trajectory generation by harnessing the complementary strengths of diffusion probabilistic models and transformers. Our proposed framework, termed the "World-Centric Diffusion Transformer"(WcDT), optimize the entire trajectory generation process. Our results show that the proposed approach exhibits superior performance in generating both realistic and diverse trajectories.
  arXiv  Detail & Related papers  (2024-04-02T16:28:41Z)
• Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models [114.69732301904419]
  We present an approach to apply end-to-end open-set (any environment/scene) autonomous driving that is capable of providing driving decisions from representations queryable by image and text. Our approach demonstrates unparalleled results in diverse tests while achieving significantly greater robustness in out-of-distribution situations.
  arXiv  Detail & Related papers  (2023-10-26T17:56:35Z)
• A Diffusion-Model of Joint Interactive Navigation [14.689298253430568]
  We present DJINN - a diffusion based method of generating traffic scenarios. Our approach jointly diffuses the trajectories of all agents, conditioned on a flexible set of state observations from the past, present, or future. We show how DJINN flexibly enables direct test-time sampling from a variety of valuable conditional distributions.
  arXiv  Detail & Related papers  (2023-09-21T22:10:20Z)
• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [149.5716746789134]
  We show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving. Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
  arXiv  Detail & Related papers  (2023-03-07T18:28:41Z)
• Guided Conditional Diffusion for Controllable Traffic Simulation [42.198185904248994]
  Controllable and realistic traffic simulation is critical for developing and verifying autonomous vehicles. Data-driven approaches generate realistic and human-like behaviors, improving transfer from simulated to real-world traffic. We develop a conditional diffusion model for controllable traffic generation (CTG) that allows users to control desired properties of trajectories at test time.
  arXiv  Detail & Related papers  (2022-10-31T14:44:59Z)
• Multi-Modal Fusion Transformer for End-to-End Autonomous Driving [59.60483620730437]
  We propose TransFuser, a novel Multi-Modal Fusion Transformer, to integrate image and LiDAR representations using attention. Our approach achieves state-of-the-art driving performance while reducing collisions by 76% compared to geometry-based fusion.
  arXiv  Detail & Related papers  (2021-04-19T11:48:13Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
  We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions. We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
  arXiv  Detail & Related papers  (2021-01-16T22:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.