JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving

• URL: http://arxiv.org/abs/2212.08710v1
• Date: Fri, 16 Dec 2022 20:59:21 GMT
• Title: JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving
• Authors: Wenjie Luo, Cheolho Park, Andre Cornman, Benjamin Sapp, Dragomir Anguelov
• Abstract summary: We propose an end-to-end trainable model that learns directly the interaction between pairs of agents in a structured, graphical model formulation. Our approach improves significantly on the trajectory overlap metrics while obtaining on-par or better performance on single-agent trajectory metrics.
• Score: 12.460224193998362
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose JFP, a Joint Future Prediction model that can learn to generate accurate and consistent multi-agent future trajectories. For this task, many different methods have been proposed to capture social interactions in the encoding part of the model, however, considerably less focus has been placed on representing interactions in the decoder and output stages. As a result, the predicted trajectories are not necessarily consistent with each other, and often result in unrealistic trajectory overlaps. In contrast, we propose an end-to-end trainable model that learns directly the interaction between pairs of agents in a structured, graphical model formulation in order to generate consistent future trajectories. It sets new state-of-the-art results on Waymo Open Motion Dataset (WOMD) for the interactive setting. We also investigate a more complex multi-agent setting for both WOMD and a larger internal dataset, where our approach improves significantly on the trajectory overlap metrics while obtaining on-par or better performance on single-agent trajectory metrics.

Related papers

• Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning [41.09061877498741]
  We propose an interaction-aware trajectory-conditioned long-term multi-agent human pose forecasting model. Our model effectively handles the multi-modality of human motion and the complexity of long-term multi-agent interactions.
  arXiv  Detail & Related papers  (2024-04-08T06:15:13Z)
• Interpretable Long Term Waypoint-Based Trajectory Prediction Model [1.4778851751964937]
  We study the impact of adding a long-term goal on the performance of a trajectory prediction framework. We present an interpretable long term waypoint-driven prediction framework (WayDCM)
  arXiv  Detail & Related papers  (2023-12-11T09:10:22Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• MotionLM: Multi-Agent Motion Forecasting as Language Modeling [15.317827804763699]
  We present MotionLM, a language model for multi-agent motion prediction. Our approach bypasses post-hoc interactions where individual agent trajectory generation is conducted prior to interactive scoring. The model's sequential factorization enables temporally causal conditional rollouts.
  arXiv  Detail & Related papers  (2023-09-28T15:46:25Z)
• MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying [110.83590008788745]
  Motion prediction is crucial for autonomous driving systems to understand complex driving scenarios and make informed decisions. In this paper, we propose Motion TRansformer (MTR) frameworks to address these challenges. The initial MTR framework utilizes a transformer encoder-decoder structure with learnable intention queries. We introduce an advanced MTR++ framework, extending the capability of MTR to simultaneously predict multimodal motion for multiple agents.
  arXiv  Detail & Related papers  (2023-06-30T16:23:04Z)
• IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction [73.25645602768158]
  IPCC-TP is a novel relevance-aware module based on Incremental Pearson Correlation Coefficient to improve multi-agent interaction modeling. Our module can be conveniently embedded into existing multi-agent prediction methods to extend original motion distribution decoders.
  arXiv  Detail & Related papers  (2023-03-01T15:16:56Z)
• FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphs [8.63314005149641]
  We propose FJMP, a Factorized Joint Motion Prediction framework for interactive driving scenarios. FJMP produces more accurate and scene-consistent joint trajectory predictions than non-factorized approaches. FJMP ranks 1st on the multi-agent test leaderboard of the INTERACTION dataset.
  arXiv  Detail & Related papers  (2022-11-27T18:59:17Z)
• THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling [2.424910201171407]
  We present a unified model architecture for fast and simultaneous agent future heatmap estimation. generating scene-consistent predictions goes beyond the mere generation of collision-free trajectories. We report our results on the Interaction multi-agent prediction challenge and rank $1st$ on the online test leaderboard.
  arXiv  Detail & Related papers  (2021-10-13T10:05:47Z)
• You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction [52.442129609979794]
  Recent deep learning approaches for trajectory prediction show promising performance. It remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions of different cues to model performance.
  arXiv  Detail & Related papers  (2021-10-11T14:24:15Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.