InfGen: Scenario Generation as Next Token Group Prediction

• URL: http://arxiv.org/abs/2506.23316v1
• Date: Sun, 29 Jun 2025 16:18:32 GMT
• Title: InfGen: Scenario Generation as Next Token Group Prediction
• Authors: Zhenghao Peng, Yuxin Liu, Bolei Zhou,
• Abstract summary: InfGen is a scenario generation framework that outputs agent states and trajectories in an autoregressive manner.<n>Experiments demonstrate that InfGen produces realistic, diverse, and adaptive traffic behaviors.
• Score: 49.54222089551598
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Realistic and interactive traffic simulation is essential for training and evaluating autonomous driving systems. However, most existing data-driven simulation methods rely on static initialization or log-replay data, limiting their ability to model dynamic, long-horizon scenarios with evolving agent populations. We propose InfGen, a scenario generation framework that outputs agent states and trajectories in an autoregressive manner. InfGen represents the entire scene as a sequence of tokens, including traffic light signals, agent states, and motion vectors, and uses a transformer model to simulate traffic over time. This design enables InfGen to continuously insert new agents into traffic, supporting infinite scene generation. Experiments demonstrate that InfGen produces realistic, diverse, and adaptive traffic behaviors. Furthermore, reinforcement learning policies trained in InfGen-generated scenarios achieve superior robustness and generalization, validating its utility as a high-fidelity simulation environment for autonomous driving. More information is available at https://metadriverse.github.io/infgen/.

Related papers

• Long-term Traffic Simulation with Interleaved Autoregressive Motion and Scenario Generation [26.98977914185036]
  An ideal traffic simulator replicates the realistic long-term point-to-point trip that a self-driving system experiences during deployment.<n>We propose InfGen, a unified next-token prediction model that performs interleaved closed-loop motion simulation and scene generation.<n>InfGen performs at the state-of-the-art in short-term (9s) traffic simulation, and significantly outperforms all other methods in long-term (30s) simulation.
  arXiv  Detail & Related papers  (2025-06-20T17:59:21Z)
• Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environments [39.649242431308544]
  We introduce Scenario Dreamer, a fully data-driven generative simulator for autonomous vehicle planning.<n>It generates both the initial traffic scene - comprising a lane graph and agent bounding boxes - and closed-loop agent behaviours.<n>Case study shows that Scenario Dreamer achieves superior generation quality with around 2x fewer parameters, 6x lower generation, and 10x training hours compared to the strongest baseline.
  arXiv  Detail & Related papers  (2025-03-28T15:03:41Z)
• RealGen: Retrieval Augmented Generation for Controllable Traffic Scenarios [58.62407014256686]
  RealGen is a novel retrieval-based in-context learning framework for traffic scenario generation. RealGen synthesizes new scenarios by combining behaviors from multiple retrieved examples in a gradient-free way. This in-context learning framework endows versatile generative capabilities, including the ability to edit scenarios.
  arXiv  Detail & Related papers  (2023-12-19T23:11:06Z)
• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [149.5716746789134]
  We show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving. Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
  arXiv  Detail & Related papers  (2023-03-07T18:28:41Z)
• DriveGAN: Towards a Controllable High-Quality Neural Simulation [147.6822288981004]
  We introduce a novel high-quality neural simulator referred to as DriveGAN. DriveGAN achieves controllability by disentangling different components without supervision. We train DriveGAN on multiple datasets, including 160 hours of real-world driving data.
  arXiv  Detail & Related papers  (2021-04-30T15:30:05Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• SceneGen: Learning to Generate Realistic Traffic Scenes [92.98412203941912]
  We present SceneGen, a neural autoregressive model of traffic scenes that eschews the need for rules and distributions. We demonstrate SceneGen's ability to faithfully model distributions of real traffic scenes.
  arXiv  Detail & Related papers  (2021-01-16T22:51:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.