BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction

• URL: http://arxiv.org/abs/2405.17372v3
• Date: Mon, 11 Nov 2024 14:20:39 GMT
• Title: BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction
• Authors: Zikang Zhou, Haibo Hu, Xinhong Chen, Jianping Wang, Nan Guan, Kui Wu, Yung-Hui Li, Yu-Kai Huang, Chun Jason Xue,
• Abstract summary: BehaviorGPT is a homogeneous and fully autoregressive Transformer designed to simulate the sequential behavior of multiple agents. We introduce the Next-Patch Prediction Paradigm (NP3) to mitigate the negative effects of autoregressive modeling. BehaviorGPT won first place in the 2024 Open Sim Agents Challenge with a realism score of 0.7473 and a minADE score of 1.4147.
• Score: 22.254486248785614
• License:
• Abstract: Simulating realistic behaviors of traffic agents is pivotal for efficiently validating the safety of autonomous driving systems. Existing data-driven simulators primarily use an encoder-decoder architecture to encode the historical trajectories before decoding the future. However, the heterogeneity between encoders and decoders complicates the models, and the manual separation of historical and future trajectories leads to low data utilization. Given these limitations, we propose BehaviorGPT, a homogeneous and fully autoregressive Transformer designed to simulate the sequential behavior of multiple agents. Crucially, our approach discards the traditional separation between "history" and "future" by modeling each time step as the "current" one for motion generation, leading to a simpler, more parameter- and data-efficient agent simulator. We further introduce the Next-Patch Prediction Paradigm (NP3) to mitigate the negative effects of autoregressive modeling, in which models are trained to reason at the patch level of trajectories and capture long-range spatial-temporal interactions. Despite having merely 3M model parameters, BehaviorGPT won first place in the 2024 Waymo Open Sim Agents Challenge with a realism score of 0.7473 and a minADE score of 1.4147, demonstrating its exceptional performance in traffic agent simulation.

Related papers

• Autonomous Vehicle Controllers From End-to-End Differentiable Simulation [60.05963742334746]
  We propose a differentiable simulator and design an analytic policy gradients (APG) approach to training AV controllers. Our proposed framework brings the differentiable simulator into an end-to-end training loop, where gradients of environment dynamics serve as a useful prior to help the agent learn a more grounded policy. We find significant improvements in performance and robustness to noise in the dynamics, as well as overall more intuitive human-like handling.
  arXiv  Detail & Related papers  (2024-09-12T11:50:06Z)
• Data-Driven Traffic Simulation for an Intersection in a Metropolis [7.264786765085108]
  We present a novel data-driven simulation environment for modeling traffic in street intersections. We train trajectory forecasting models to learn agent interactions and environmental constraints. The simulation can run either autonomously, or under explicit human control conditioned on the generative distributions.
  arXiv  Detail & Related papers  (2024-08-01T22:25:06Z)
• Model Predictive Simulation Using Structured Graphical Models and Transformers [4.229560419171488]
  We propose an approach to simulating trajectories of multiple interacting agents (road users) based on transformers and probabilistic graphical models (PGMs) We then improve upon these generated trajectories using a PGM, which contains factors which encode prior knowledge. We show that MPS improves upon the MTR baseline, especially in safety critical metrics such as collision rate.
  arXiv  Detail & Related papers  (2024-06-28T03:46:53Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Deciphering Movement: Unified Trajectory Generation Model for Multi-Agent [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs. Specifically, we introduce a Ghost Spatial Masking (GSM) module embedded within a Transformer encoder for spatial feature extraction. We benchmark three practical sports game datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)
• SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction [4.318757942343036]
  We introduce a novel autonomous driving motion generation paradigm that models vectorized map and agent trajectory data into discrete sequence tokens. These tokens are then processed through a decoder-only transformer architecture to train for the next token prediction task. We have collected over 1 billion motion tokens from multiple datasets, validating the model's scalability.
  arXiv  Detail & Related papers  (2024-05-24T16:17:35Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• STGlow: A Flow-based Generative Framework with Dual Graphormer for Pedestrian Trajectory Prediction [22.553356096143734]
  We propose a novel generative flow based framework with dual graphormer for pedestrian trajectory prediction (STGlow) Our method can more precisely model the underlying data distribution by optimizing the exact log-likelihood of motion behaviors. Experimental results on several benchmarks demonstrate that our method achieves much better performance compared to previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-11-21T07:29:24Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.