Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion

• URL: http://arxiv.org/abs/2203.02489v1
• Date: Fri, 4 Mar 2022 18:39:31 GMT
• Title: Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion
• Authors: Dongxu Guo, Taylor Mordan, Alexandre Alahi
• Abstract summary: We introduce the new task of pedestrian stop and go forecasting. Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic. We build it from several existing datasets annotated with pedestrians' walking motions, in order to have various scenarios and behaviors.
• Score: 87.77727495366702
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Forecasting pedestrians' future motions is essential for autonomous driving systems to safely navigate in urban areas. However, existing prediction algorithms often overly rely on past observed trajectories and tend to fail around abrupt dynamic changes, such as when pedestrians suddenly start or stop walking. We suggest that predicting these highly non-linear transitions should form a core component to improve the robustness of motion prediction algorithms. In this paper, we introduce the new task of pedestrian stop and go forecasting. Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic. We build it from several existing datasets annotated with pedestrians' walking motions, in order to have various scenarios and behaviors. We also propose a novel hybrid model that leverages pedestrian-specific and scene features from several modalities, both video sequences and high-level attributes, and gradually fuses them to integrate multiple levels of context. We evaluate our model and several baselines on TRANS, and set a new benchmark for the community to work on pedestrian stop and go forecasting.

Related papers

• Social-Transmotion: Promptable Human Trajectory Prediction [65.80068316170613]
  Social-Transmotion is a generic Transformer-based model that exploits diverse and numerous visual cues to predict human behavior. Our approach is validated on multiple datasets, including JTA, JRDB, Pedestrians and Cyclists in Road Traffic, and ETH-UCY.
  arXiv  Detail & Related papers  (2023-12-26T18:56:49Z)
• Pedestrian Trajectory Prediction via Spatial Interaction Transformer Network [7.150832716115448]
  In traffic scenes, when encountering with oncoming people, pedestrians may make sudden turns or stop immediately. To predict such unpredictable trajectories, we can gain insights into the interaction between pedestrians. We present a novel generative method named Spatial Interaction Transformer (SIT), which learns the correlation of pedestrian trajectories through attention mechanisms.
  arXiv  Detail & Related papers  (2021-12-13T13:08:04Z)
• Learning Sparse Interaction Graphs of Partially Observed Pedestrians for Trajectory Prediction [0.3025231207150811]
  Multi-pedestrian trajectory prediction is an indispensable safety element of autonomous systems that interact with crowds in unstructured environments. We propose Gumbel Social Transformer, in which an Edge Gumbel Selector samples a sparse graph of partially observed pedestrians at each time step. We demonstrate that our model overcomes the potential problems caused by the assumptions, and our approach outperforms the related works in benchmark evaluation.
  arXiv  Detail & Related papers  (2021-07-15T00:45:11Z)
• Safety-Oriented Pedestrian Motion and Scene Occupancy Forecasting [91.69900691029908]
  We advocate for predicting both the individual motions as well as the scene occupancy map. We propose a Scene-Actor Graph Neural Network (SA-GNN) which preserves the relative spatial information of pedestrians. On two large-scale real-world datasets, we showcase that our scene-occupancy predictions are more accurate and better calibrated than those from state-of-the-art motion forecasting methods.
  arXiv  Detail & Related papers  (2021-01-07T06:08:21Z)
• PePScenes: A Novel Dataset and Baseline for Pedestrian Action Prediction in 3D [10.580548257913843]
  We propose a new pedestrian action prediction dataset created by adding per-frame 2D/3D bounding box and behavioral annotations to nuScenes. In addition, we propose a hybrid neural network architecture that incorporates various data modalities for predicting pedestrian crossing action.
  arXiv  Detail & Related papers  (2020-12-14T18:13:44Z)
• Graph-SIM: A Graph-based Spatiotemporal Interaction Modelling for Pedestrian Action Prediction [10.580548257913843]
  We propose a novel graph-based model for predicting pedestrian crossing action. We introduce a new dataset that provides 3D bounding box and pedestrian behavioural annotations for the existing nuScenes dataset. Our approach achieves state-of-the-art performance by improving on various metrics by more than 15% in comparison to existing methods.
  arXiv  Detail & Related papers  (2020-12-03T18:28:27Z)
• Multi-Modal Hybrid Architecture for Pedestrian Action Prediction [14.032334569498968]
  We propose a novel multi-modal prediction algorithm that incorporates different sources of information captured from the environment to predict future crossing actions of pedestrians. Using the existing 2D pedestrian behavior benchmarks and a newly annotated 3D driving dataset, we show that our proposed model achieves state-of-the-art performance in pedestrian crossing prediction.
  arXiv  Detail & Related papers  (2020-11-16T15:17:58Z)
• Pedestrian Intention Prediction: A Multi-task Perspective [83.7135926821794]
  In order to be globally deployed, autonomous cars must guarantee the safety of pedestrians. This work tries to solve this problem by jointly predicting the intention and visual states of pedestrians. The method is a recurrent neural network in a multi-task learning approach.
  arXiv  Detail & Related papers  (2020-10-20T13:42:31Z)
• TPNet: Trajectory Proposal Network for Motion Prediction [81.28716372763128]
  Trajectory Proposal Network (TPNet) is a novel two-stage motion prediction framework. TPNet first generates a candidate set of future trajectories as hypothesis proposals, then makes the final predictions by classifying and refining the proposals. Experiments on four large-scale trajectory prediction datasets, show that TPNet achieves the state-of-the-art results both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2020-04-26T00:01:49Z)
• Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction [57.56466850377598]
  Reasoning over visual data is a desirable capability for robotics and vision-based applications. In this paper, we present a framework on graph to uncover relationships in different objects in the scene for reasoning about pedestrian intent. Pedestrian intent, defined as the future action of crossing or not-crossing the street, is a very crucial piece of information for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-02-20T18:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.