AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving

• URL: http://arxiv.org/abs/2403.13331v2
• Date: Thu, 21 Mar 2024 04:01:10 GMT
• Title: AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving
• Authors: Xiaosong Jia, Shaoshuai Shi, Zijun Chen, Li Jiang, Wenlong Liao, Tao He, Junchi Yan,
• Abstract summary: We introduce the GPT style next token motion prediction into motion prediction. Different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations. We propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations.
• Score: 59.94343412438211
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: As an essential task in autonomous driving (AD), motion prediction aims to predict the future states of surround objects for navigation. One natural solution is to estimate the position of other agents in a step-by-step manner where each predicted time-step is conditioned on both observed time-steps and previously predicted time-steps, i.e., autoregressive prediction. Pioneering works like SocialLSTM and MFP design their decoders based on this intuition. However, almost all state-of-the-art works assume that all predicted time-steps are independent conditioned on observed time-steps, where they use a single linear layer to generate positions of all time-steps simultaneously. They dominate most motion prediction leaderboards due to the simplicity of training MLPs compared to autoregressive networks. In this paper, we introduce the GPT style next token prediction into motion forecasting. In this way, the input and output could be represented in a unified space and thus the autoregressive prediction becomes more feasible. However, different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations. To this end, we propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations, e.g., encoding the transformation between coordinate systems for spatial relativity while adopting RoPE for temporal relativity. Empirically, by equipping with the aforementioned tailored designs, the proposed method achieves state-of-the-art performance in the Waymo Open Motion and Waymo Interaction datasets. Notably, AMP outperforms other recent autoregressive motion prediction methods: MotionLM and StateTransformer, which demonstrates the effectiveness of the proposed designs.

Related papers

• Motion Forecasting in Continuous Driving [41.6423398623095]
  In autonomous driving, motion forecasting takes place repeatedly and continuously as the self-driving car moves. Existing forecasting methods process each driving scene within a certain range independently. We propose a novel motion forecasting framework for continuous driving, named RealMotion.
  arXiv  Detail & Related papers  (2024-10-08T13:04:57Z)
• PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving [57.89801036693292]
  PPAD (Iterative Interaction of Prediction and Planning Autonomous Driving) considers the timestep-wise interaction to better integrate prediction and planning. We design ego-to-agent, ego-to-map, and ego-to-BEV interaction mechanisms with hierarchical dynamic key objects attention to better model the interactions.
  arXiv  Detail & Related papers  (2023-11-14T11:53:24Z)
• CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion [6.862357145175449]
  We propose CoMusion, a single-stage, end-to-end diffusion-based HMP framework. CoMusion is inspired from the insight that a smooth future pose prediction performance improves spatial prediction performance. Our method, facilitated by the Transformer-GCN module design and a proposed variance scheduler, predicts accurate, realistic, and consistent motions.
  arXiv  Detail & Related papers  (2023-05-21T19:31:56Z)
• A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Prediction [4.181632607997678]
  We propose a hierarchical hybrid framework of deep learning (DL) and reinforcement learning (RL) for multi-agent trajectory prediction. In the DL stage, the traffic scene is divided into multiple intermediate-scale heterogenous graphs based on which Transformer-style GNNs are adopted to encode heterogenous interactions. In the RL stage, we divide the traffic scene into local sub-scenes utilizing the key future points predicted in the DL stage.
  arXiv  Detail & Related papers  (2023-03-22T02:47:42Z)
• Motion Transformer with Global Intention Localization and Local Movement Refinement [103.75625476231401]
  Motion TRansformer (MTR) models motion prediction as the joint optimization of global intention localization and local movement refinement. MTR achieves state-of-the-art performance on both the marginal and joint motion prediction challenges.
  arXiv  Detail & Related papers  (2022-09-27T16:23:14Z)
• Exploring Attention GAN for Vehicle Motion Prediction [2.887073662645855]
  We study the influence of attention in generative models for motion prediction, considering both physical and social context. We validate our method using the Argoverse Motion Forecasting Benchmark 1.1, achieving competitive unimodal results.
  arXiv  Detail & Related papers  (2022-09-26T13:18:32Z)
• Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion [88.45326906116165]
  We present a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID) We encode the history behavior information and the social interactions as a state embedding and devise a Transformer-based diffusion model to capture the temporal dependencies of trajectories. Experiments on the human trajectory prediction benchmarks including the Stanford Drone and ETH/UCY datasets demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-03-25T16:59:08Z)
• Motion Prediction Using Temporal Inception Module [96.76721173517895]
  We propose a Temporal Inception Module (TIM) to encode human motion. Our framework produces input embeddings using convolutional layers, by using different kernel sizes for different input lengths. The experimental results on standard motion prediction benchmark datasets Human3.6M and CMU motion capture dataset show that our approach consistently outperforms the state of the art methods.
  arXiv  Detail & Related papers  (2020-10-06T20:26:01Z)
• Implicit Latent Variable Model for Scene-Consistent Motion Forecasting [78.74510891099395]
  In this paper, we aim to learn scene-consistent motion forecasts of complex urban traffic directly from sensor data. We model the scene as an interaction graph and employ powerful graph neural networks to learn a distributed latent representation of the scene.
  arXiv  Detail & Related papers  (2020-07-23T14:31:25Z)
• AMENet: Attentive Maps Encoder Network for Trajectory Prediction [35.22312783822563]
  Trajectory prediction is critical for applications of planning safe future movements. We propose an end-to-end generative model named Attentive Maps Network (AMENet) AMENet encodes the agent's motion and interaction information for accurate and realistic multi-path trajectory prediction.
  arXiv  Detail & Related papers  (2020-06-15T10:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.