Model Predictive Simulation Using Structured Graphical Models and Transformers

• URL: http://arxiv.org/abs/2406.19635v1
• Date: Fri, 28 Jun 2024 03:46:53 GMT
• Title: Model Predictive Simulation Using Structured Graphical Models and Transformers
• Authors: Xinghua Lou, Meet Dave, Shrinu Kushagra, Miguel Lazaro-Gredilla, Kevin Murphy,
• Abstract summary: We propose an approach to simulating trajectories of multiple interacting agents (road users) based on transformers and probabilistic graphical models (PGMs) We then improve upon these generated trajectories using a PGM, which contains factors which encode prior knowledge. We show that MPS improves upon the MTR baseline, especially in safety critical metrics such as collision rate.
• Score: 4.229560419171488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose an approach to simulating trajectories of multiple interacting agents (road users) based on transformers and probabilistic graphical models (PGMs), and apply it to the Waymo SimAgents challenge. The transformer baseline is based on the MTR model, which predicts multiple future trajectories conditioned on the past trajectories and static road layout features. We then improve upon these generated trajectories using a PGM, which contains factors which encode prior knowledge, such as a preference for smooth trajectories, and avoidance of collisions with static obstacles and other moving agents. We perform (approximate) MAP inference in this PGM using the Gauss-Newton method. Finally we sample $K=32$ trajectories for each of the $N \sim 100$ agents for the next $T=8 \Delta$ time steps, where $\Delta=10$ is the sampling rate per second. Following the Model Predictive Control (MPC) paradigm, we only return the first element of our forecasted trajectories at each step, and then we replan, so that the simulation can constantly adapt to its changing environment. We therefore call our approach "Model Predictive Simulation" or MPS. We show that MPS improves upon the MTR baseline, especially in safety critical metrics such as collision rate. Furthermore, our approach is compatible with any underlying forecasting model, and does not require extra training, so we believe it is a valuable contribution to the community.

Related papers

• Data-Driven Traffic Simulation for an Intersection in a Metropolis [7.264786765085108]
  We present a novel data-driven simulation environment for modeling traffic in street intersections. We train trajectory forecasting models to learn agent interactions and environmental constraints. The simulation can run either autonomously, or under explicit human control conditioned on the generative distributions.
  arXiv  Detail & Related papers  (2024-08-01T22:25:06Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  Motion planners (MPs) are crucial for safe navigation in complex urban environments. nuPlan, a recently released MP benchmark, addresses this limitation by augmenting real-world driving logs with closed-loop simulation logic. We present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Prediction [22.254486248785614]
  Behavior Generative Pre-trained Transformers (BehaviorGPT) is a decoder-only, autoregressive architecture designed to simulate the sequential motion of multiple agents. Next-Patch Prediction Paradigm (NP3) enables models to reason at the patch level of trajectories and capture long-range spatial-temporal interactions. BehaviorGPT ranks first across several metrics on the Sim Agents Benchmark, demonstrating its exceptional performance in multi-agent and agent-map interactions.
  arXiv  Detail & Related papers  (2024-05-27T17:28:25Z)
• Rocket Landing Control with Grid Fins and Path-following using MPC [0.0]
  The goal is to minimize the total fuel consumption during the landing process using different techniques. Once the optimal and feasible trajectory is generated using batch approach, we attempt to follow the path using a Model Predictive Control (MPC) based algorithm. We show that TOPED can follow a demonstration trajectory well in practice under model mismatch and different initial states.
  arXiv  Detail & Related papers  (2024-05-25T11:42:29Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• Pre-training on Synthetic Driving Data for Trajectory Prediction [61.520225216107306]
  We propose a pipeline-level solution to mitigate the issue of data scarcity in trajectory forecasting. We adopt HD map augmentation and trajectory synthesis for generating driving data, and then we learn representations by pre-training on them. We conduct extensive experiments to demonstrate the effectiveness of our data expansion and pre-training strategies.
  arXiv  Detail & Related papers  (2023-09-18T19:49:22Z)
• Emergent Agentic Transformer from Chain of Hindsight Experience [96.56164427726203]
  We show that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches. This is the first time that a simple transformer-based model performs competitively with both temporal-difference and imitation-learning-based approaches.
  arXiv  Detail & Related papers  (2023-05-26T00:43:02Z)
• THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling [2.424910201171407]
  We present a unified model architecture for fast and simultaneous agent future heatmap estimation. generating scene-consistent predictions goes beyond the mere generation of collision-free trajectories. We report our results on the Interaction multi-agent prediction challenge and rank $1st$ on the online test leaderboard.
  arXiv  Detail & Related papers  (2021-10-13T10:05:47Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.