Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation

• URL: http://arxiv.org/abs/2104.11212v1
• Date: Thu, 22 Apr 2021 17:48:08 GMT
• Title: Imagining The Road Ahead: Multi-Agent Trajectory Prediction via Differentiable Simulation
• Authors: Adam Scibior, Vasileios Lioutas, Daniele Reda, Peyman Bateni, Frank Wood
• Abstract summary: We develop a deep generative model built on a fully differentiable simulator for trajectory prediction. We achieve state-of-the-art results on the INTERACTION dataset, using standard neural architectures and a standard variational training objective. We name our model ITRA, for "Imagining the Road Ahead"
• Score: 17.953880589741438
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We develop a deep generative model built on a fully differentiable simulator for multi-agent trajectory prediction. Agents are modeled with conditional recurrent variational neural networks (CVRNNs), which take as input an ego-centric birdview image representing the current state of the world and output an action, consisting of steering and acceleration, which is used to derive the subsequent agent state using a kinematic bicycle model. The full simulation state is then differentiably rendered for each agent, initiating the next time step. We achieve state-of-the-art results on the INTERACTION dataset, using standard neural architectures and a standard variational training objective, producing realistic multi-modal predictions without any ad-hoc diversity-inducing losses. We conduct ablation studies to examine individual components of the simulator, finding that both the kinematic bicycle model and the continuous feedback from the birdview image are crucial for achieving this level of performance. We name our model ITRA, for "Imagining the Road Ahead".

Related papers

• Data-Driven Traffic Simulation for an Intersection in a Metropolis [7.264786765085108]
  We present a novel data-driven simulation environment for modeling traffic in street intersections. We train trajectory forecasting models to learn agent interactions and environmental constraints. The simulation can run either autonomously, or under explicit human control conditioned on the generative distributions.
  arXiv  Detail & Related papers  (2024-08-01T22:25:06Z)
• Vectorized Representation Dreamer (VRD): Dreaming-Assisted Multi-Agent Motion-Forecasting [2.2020053359163305]
  We introduce VRD, a vectorized world model-inspired approach to the multi-agent motion forecasting problem. Our method combines a traditional open-loop training regime with a novel dreamed closed-loop training pipeline. Our model achieves state-of-the-art performance on the single prediction miss rate metric.
  arXiv  Detail & Related papers  (2024-06-20T15:34:17Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Research [76.93956925360638]
  Waymax is a new data-driven simulator for autonomous driving in multi-agent scenes. It runs entirely on hardware accelerators such as TPUs/GPUs and supports in-graph simulation for training. We benchmark a suite of popular imitation and reinforcement learning algorithms with ablation studies on different design decisions.
  arXiv  Detail & Related papers  (2023-10-12T20:49:15Z)
• TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction [149.5716746789134]
  We show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving. Experiments on the open motion dataset show TrafficBots can simulate realistic multi-agent behaviors.
  arXiv  Detail & Related papers  (2023-03-07T18:28:41Z)
• TrajGen: Generating Realistic and Diverse Trajectories with Reactive and Feasible Agent Behaviors for Autonomous Driving [19.06020265777298]
  Existing simulators rely on system-based behavior models for background vehicles, which cannot capture the complex interactive behaviors in real-world scenarios. We propose TrajGen, a two-stage trajectory generation framework, which can capture more realistic behaviors directly from human demonstration. In addition, we develop a data-driven simulator I-Sim that can be used to train reinforcement learning models in parallel based on naturalistic driving data.
  arXiv  Detail & Related papers  (2022-03-31T04:48:29Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors [74.67698916175614]
  We propose TrafficSim, a multi-agent behavior model for realistic traffic simulation. In particular, we leverage an implicit latent variable model to parameterize a joint actor policy. We show TrafficSim generates significantly more realistic and diverse traffic scenarios as compared to a diverse set of baselines.
  arXiv  Detail & Related papers  (2021-01-17T00:29:30Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.