PMM-Net: Single-stage Multi-agent Trajectory Prediction with Patching-based Embedding and Explicit Modal Modulation

• URL: http://arxiv.org/abs/2410.19544v1
• Date: Fri, 25 Oct 2024 13:16:27 GMT
• Title: PMM-Net: Single-stage Multi-agent Trajectory Prediction with Patching-based Embedding and Explicit Modal Modulation
• Authors: Huajian Liu, Wei Dong, Kunpeng Fan, Chao Wang, Yongzhuo Gao,
• Abstract summary: In this letter, we aim to explore a distinct formulation for multi-agent trajectory prediction framework. We propose a patching-based temporal feature extraction module and a graph-based social feature extraction module. We present a novel method based on explicit modality modulation to integrate temporal and social features, thereby constructing an efficient single-stage inference pipeline.
• Score: 6.793915571620126
• License:
• Abstract: Analyzing and forecasting trajectories of agents like pedestrians plays a pivotal role for embodied intelligent applications. The inherent indeterminacy of human behavior and complex social interaction among a rich variety of agents make this task more challenging than common time-series forecasting. In this letter, we aim to explore a distinct formulation for multi-agent trajectory prediction framework. Specifically, we proposed a patching-based temporal feature extraction module and a graph-based social feature extraction module, enabling effective feature extraction and cross-scenario generalization. Moreover, we reassess the role of social interaction and present a novel method based on explicit modality modulation to integrate temporal and social features, thereby constructing an efficient single-stage inference pipeline. Results on public benchmark datasets demonstrate the superior performance of our model compared with the state-of-the-art methods. The code is available at: github.com/TIB-K330/pmm-net.

Related papers

• Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning [51.52387511006586]
  We propose Hierarchical Opponent modeling and Planning (HOP), a novel multi-agent decision-making algorithm. HOP is hierarchically composed of two modules: an opponent modeling module that infers others' goals and learns corresponding goal-conditioned policies. HOP exhibits superior few-shot adaptation capabilities when interacting with various unseen agents, and excels in self-play scenarios.
  arXiv  Detail & Related papers  (2024-06-12T08:48:06Z)
• A Large-Scale Evaluation of Speech Foundation Models [110.95827399522204]
  We establish the Speech processing Universal PERformance Benchmark (SUPERB) to study the effectiveness of the foundation model paradigm for speech. We propose a unified multi-tasking framework to address speech processing tasks in SUPERB using a frozen foundation model followed by task-specialized, lightweight prediction heads.
  arXiv  Detail & Related papers  (2024-04-15T00:03:16Z)
• Cheap and Deterministic Inference for Deep State-Space Models of Interacting Dynamical Systems [38.23826389188657]
  We present a deep state-space model which employs graph neural networks in order to model the underlying interacting dynamical system. The predictive distribution is multimodal and has the form of a Gaussian mixture model, where the moments of the Gaussian components can be computed via deterministic moment matching rules. Our moment matching scheme can be exploited for sample-free inference, leading to more efficient and stable training compared to Monte Carlo alternatives.
  arXiv  Detail & Related papers  (2023-05-02T20:30:23Z)
• JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving [12.460224193998362]
  We propose an end-to-end trainable model that learns directly the interaction between pairs of agents in a structured, graphical model formulation. Our approach improves significantly on the trajectory overlap metrics while obtaining on-par or better performance on single-agent trajectory metrics.
  arXiv  Detail & Related papers  (2022-12-16T20:59:21Z)
• Conditioned Human Trajectory Prediction using Iterative Attention Blocks [70.36888514074022]
  We present a simple yet effective pedestrian trajectory prediction model aimed at pedestrians positions prediction in urban-like environments. Our model is a neural-based architecture that can run several layers of attention blocks and transformers in an iterative sequential fashion. We show that without explicit introduction of social masks, dynamical models, social pooling layers, or complicated graph-like structures, it is possible to produce on par results with SoTA models.
  arXiv  Detail & Related papers  (2022-06-29T07:49:48Z)
• Goal-driven Self-Attentive Recurrent Networks for Trajectory Prediction [31.02081143697431]
  Human trajectory forecasting is a key component of autonomous vehicles, social-aware robots and video-surveillance applications. We propose a lightweight attention-based recurrent backbone that acts solely on past observed positions. We employ a common goal module, based on a U-Net architecture, which additionally extracts semantic information to predict scene-compliant destinations.
  arXiv  Detail & Related papers  (2022-04-25T11:12:37Z)
• Exploring Social Posterior Collapse in Variational Autoencoder for Interaction Modeling [26.01824780050843]
  Variational Autoencoder (VAE) has been widely applied in multi-agent interaction modeling. VAE is prone to ignoring historical social context when predicting the future trajectory of an agent. We propose a novel sparse graph attention message-passing layer, which helps us detect social posterior collapse.
  arXiv  Detail & Related papers  (2021-12-01T06:20:58Z)
• You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction [52.442129609979794]
  Recent deep learning approaches for trajectory prediction show promising performance. It remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions of different cues to model performance.
  arXiv  Detail & Related papers  (2021-10-11T14:24:15Z)
• Multi-Agent Imitation Learning with Copulas [102.27052968901894]
  Multi-agent imitation learning aims to train multiple agents to perform tasks from demonstrations by learning a mapping between observations and actions. In this paper, we propose to use copula, a powerful statistical tool for capturing dependence among random variables, to explicitly model the correlation and coordination in multi-agent systems. Our proposed model is able to separately learn marginals that capture the local behavioral patterns of each individual agent, as well as a copula function that solely and fully captures the dependence structure among agents.
  arXiv  Detail & Related papers  (2021-07-10T03:49:41Z)
• Instance-Aware Predictive Navigation in Multi-Agent Environments [93.15055834395304]
  We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures. We adopt a novel multi-instance event prediction module to estimate the possible interaction among agents in the ego-centric view. We design a sequential action sampling strategy to better leverage predicted states on both scene-level and instance-level.
  arXiv  Detail & Related papers  (2021-01-14T22:21:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.