A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Prediction

• URL: http://arxiv.org/abs/2303.12274v3
• Date: Fri, 24 Mar 2023 07:05:08 GMT
• Title: A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Prediction
• Authors: Yujun Jiao, Mingze Miao, Zhishuai Yin, Chunyuan Lei, Xu Zhu, Linzhen Nie and Bo Tao
• Abstract summary: We propose a hierarchical hybrid framework of deep learning (DL) and reinforcement learning (RL) for multi-agent trajectory prediction. In the DL stage, the traffic scene is divided into multiple intermediate-scale heterogenous graphs based on which Transformer-style GNNs are adopted to encode heterogenous interactions. In the RL stage, we divide the traffic scene into local sub-scenes utilizing the key future points predicted in the DL stage.
• Score: 4.181632607997678
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurate and robust trajectory prediction of neighboring agents is critical for autonomous vehicles traversing in complex scenes. Most methods proposed in recent years are deep learning-based due to their strength in encoding complex interactions. However, unplausible predictions are often generated since they rely heavily on past observations and cannot effectively capture the transient and contingency interactions from sparse samples. In this paper, we propose a hierarchical hybrid framework of deep learning (DL) and reinforcement learning (RL) for multi-agent trajectory prediction, to cope with the challenge of predicting motions shaped by multi-scale interactions. In the DL stage, the traffic scene is divided into multiple intermediate-scale heterogenous graphs based on which Transformer-style GNNs are adopted to encode heterogenous interactions at intermediate and global levels. In the RL stage, we divide the traffic scene into local sub-scenes utilizing the key future points predicted in the DL stage. To emulate the motion planning procedure so as to produce trajectory predictions, a Transformer-based Proximal Policy Optimization (PPO) incorporated with a vehicle kinematics model is devised to plan motions under the dominant influence of microscopic interactions. A multi-objective reward is designed to balance between agent-centric accuracy and scene-wise compatibility. Experimental results show that our proposal matches the state-of-the-arts on the Argoverse forecasting benchmark. It's also revealed by the visualized results that the hierarchical learning framework captures the multi-scale interactions and improves the feasibility and compliance of the predicted trajectories.

Related papers

• AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving [59.94343412438211]
  We introduce the GPT style next token motion prediction into motion prediction. Different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations. We propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations.
  arXiv  Detail & Related papers  (2024-03-20T06:22:37Z)
• Multi-Agent Dynamic Relational Reasoning for Social Robot Navigation [50.01551945190676]
  Social robot navigation can be helpful in various contexts of daily life but requires safe human-robot interactions and efficient trajectory planning. We propose a systematic relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures. We demonstrate its effectiveness for multi-agent trajectory prediction and social robot navigation.
  arXiv  Detail & Related papers  (2024-01-22T18:58:22Z)
• Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding [121.08841110022607]
  Existing agent-centric methods have demonstrated outstanding performance on public benchmarks. We introduce the K-nearest neighbor attention with relative pose encoding (KNARPE), a novel attention mechanism allowing the pairwise-relative representation to be used by Transformers. By sharing contexts among agents and reusing the unchanged contexts, our approach is as efficient as scene-centric methods, while performing on par with state-of-the-art agent-centric methods.
  arXiv  Detail & Related papers  (2023-10-19T17:59:01Z)
• IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction [73.25645602768158]
  IPCC-TP is a novel relevance-aware module based on Incremental Pearson Correlation Coefficient to improve multi-agent interaction modeling. Our module can be conveniently embedded into existing multi-agent prediction methods to extend original motion distribution decoders.
  arXiv  Detail & Related papers  (2023-03-01T15:16:56Z)
• LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction [12.84508682310717]
  We propose LatentFormer, a transformer-based model for predicting future vehicle trajectories. We evaluate the proposed method on the nuScenes benchmark dataset and show that our approach achieves state-of-the-art performance and improves upon trajectory metrics by up to 40%.
  arXiv  Detail & Related papers  (2022-03-03T17:44:58Z)
• SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction [64.16212996247943]
  We present a Sparse Graph Convolution Network(SGCN) for pedestrian trajectory prediction. Specifically, the SGCN explicitly models the sparse directed interaction with a sparse directed spatial graph to capture adaptive interaction pedestrians. visualizations indicate that our method can capture adaptive interactions between pedestrians and their effective motion tendencies.
  arXiv  Detail & Related papers  (2021-04-04T03:17:42Z)
• Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking [23.608125748229174]
  We propose a generic generative neural system for multi-agent trajectory prediction involving heterogeneous agents. The proposed system is evaluated on three public benchmark datasets for trajectory prediction.
  arXiv  Detail & Related papers  (2021-02-18T02:25:35Z)
• Multimodal Trajectory Prediction via Topological Invariance for Navigation at Uncontrolled Intersections [45.508973373913946]
  We focus on decentralized navigation among multiple non-communicating rational agents at street intersections without traffic signs or signals. Our key insight is that the geometric structure of the intersection and the incentive of agents to move efficiently and avoid collisions (rationality) reduces the space of likely behaviors. We design Multiple Topologies Prediction (MTP), a data-driven trajectory-prediction mechanism that reconstructs trajectory representations of high-likelihood modes in multiagent intersection scenes.
  arXiv  Detail & Related papers  (2020-11-08T02:56:42Z)
• Implicit Latent Variable Model for Scene-Consistent Motion Forecasting [78.74510891099395]
  In this paper, we aim to learn scene-consistent motion forecasts of complex urban traffic directly from sensor data. We model the scene as an interaction graph and employ powerful graph neural networks to learn a distributed latent representation of the scene.
  arXiv  Detail & Related papers  (2020-07-23T14:31:25Z)
• Robust Trajectory Forecasting for Multiple Intelligent Agents in Dynamic Scene [11.91073327154494]
  We present a novel method for robust trajectory forecasting of multiple agents in dynamic scenes. The proposed method outperforms the state-of-the-art prediction methods in terms of prediction accuracy.
  arXiv  Detail & Related papers  (2020-05-27T02:32:55Z)
• Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network [29.289670231364788]
  In this paper, we propose a generic generative neural system for multi-agent trajectory prediction. We also employ an efficient kinematic constraint layer applied to vehicle trajectory prediction. The proposed system is evaluated on three public benchmark datasets for trajectory prediction.
  arXiv  Detail & Related papers  (2020-02-14T20:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.