FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network

• URL: http://arxiv.org/abs/2005.07796v1
• Date: Fri, 15 May 2020 21:52:42 GMT
• Title: FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network
• Authors: Francesco Piccoli, Rajarathnam Balakrishnan, Maria Jesus Perez, Moraldeepsingh Sachdeo, Carlos Nunez, Matthew Tang, Kajsa Andreasson, Kalle Bjurek, Ria Dass Raj, Ebba Davidsson, Colin Eriksson, Victor Hagman, Jonas Sjoberg, Ying Li, L. Srikar Muppirisetty, Sohini Roychowdhury
• Abstract summary: We develop an end-to-end pedestrian intention framework that performs well on day- and night-time scenarios. Our framework relies on objection detection bounding boxes combined with skeletal features of human pose.
• Score: 3.5581822321535785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pedestrian intention recognition is very important to develop robust and safe autonomous driving (AD) and advanced driver assistance systems (ADAS) functionalities for urban driving. In this work, we develop an end-to-end pedestrian intention framework that performs well on day- and night- time scenarios. Our framework relies on objection detection bounding boxes combined with skeletal features of human pose. We study early, late, and combined (early and late) fusion mechanisms to exploit the skeletal features and reduce false positives as well to improve the intention prediction performance. The early fusion mechanism results in AP of 0.89 and precision/recall of 0.79/0.89 for pedestrian intention classification. Furthermore, we propose three new metrics to properly evaluate the pedestrian intention systems. Under these new evaluation metrics for the intention prediction, the proposed end-to-end network offers accurate pedestrian intention up to half a second ahead of the actual risky maneuver.

Related papers

• Valeo4Cast: A Modular Approach to End-to-End Forecasting [93.86257326005726]
  Our solution ranks first in the Argoverse 2 End-to-end Forecasting Challenge, with 63.82 mAPf. We depart from the current trend of tackling this task via end-to-end training from perception to forecasting, and instead use a modular approach. We surpass forecasting results by +17.1 points over last year's winner and by +13.3 points over this year's runner-up.
  arXiv  Detail & Related papers  (2024-06-12T11:50:51Z)
• AdvDO: Realistic Adversarial Attacks for Trajectory Prediction [87.96767885419423]
  Trajectory prediction is essential for autonomous vehicles to plan correct and safe driving behaviors. We devise an optimization-based adversarial attack framework to generate realistic adversarial trajectories. Our attack can lead an AV to drive off road or collide into other vehicles in simulation.
  arXiv  Detail & Related papers  (2022-09-19T03:34:59Z)
• Pedestrian 3D Bounding Box Prediction [83.7135926821794]
  We focus on 3D bounding boxes, which are reasonable estimates of humans without modeling complex motion details for autonomous vehicles. We suggest this new problem and present a simple yet effective model for pedestrians' 3D bounding box prediction. This method follows an encoder-decoder architecture based on recurrent neural networks.
  arXiv  Detail & Related papers  (2022-06-28T17:59:45Z)
• Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion [87.77727495366702]
  We introduce the new task of pedestrian stop and go forecasting. Considering the lack of suitable existing datasets for it, we release TRANS, a benchmark for explicitly studying the stop and go behaviors of pedestrians in urban traffic. We build it from several existing datasets annotated with pedestrians' walking motions, in order to have various scenarios and behaviors.
  arXiv  Detail & Related papers  (2022-03-04T18:39:31Z)
• TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction [1.911678487931003]
  We address pedestrian crossing prediction in urban traffic environments by linking the dynamics of a pedestrian's skeleton to a binary crossing intention. We introduce TrouSPI-Net: a context-free, lightweight, lightweight predictor. We evaluate TrouSPI-Net and analyze its performance.
  arXiv  Detail & Related papers  (2021-09-02T13:54:02Z)
• Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention [0.0]
  Pedestrian crossing intention should be recognized in real-time for urban driving. Recent works have shown the potential of using vision-based deep neural network models for this task. This work introduces a neural network architecture to fuse inherently different novel-temporal features for pedestrian crossing intention prediction.
  arXiv  Detail & Related papers  (2021-04-12T14:10:25Z)
• IntentNet: Learning to Predict Intention from Raw Sensor Data [86.74403297781039]
  In this paper, we develop a one-stage detector and forecaster that exploits both 3D point clouds produced by a LiDAR sensor as well as dynamic maps of the environment. Our multi-task model achieves better accuracy than the respective separate modules while saving computation, which is critical to reducing reaction time in self-driving applications.
  arXiv  Detail & Related papers  (2021-01-20T00:31:52Z)
• Pedestrian Behavior Prediction for Automated Driving: Requirements, Metrics, and Relevant Features [1.1888947789336193]
  We analyze the requirements on pedestrian behavior prediction for automated driving via a system-level approach. Based on human driving behavior we derive appropriate reaction patterns of an automated vehicle. We present a pedestrian prediction model based on a Variational Conditional Auto-Encoder which incorporates multiple contextual cues.
  arXiv  Detail & Related papers  (2020-12-15T16:52:49Z)
• A Real-Time Predictive Pedestrian Collision Warning Service for Cooperative Intelligent Transportation Systems Using 3D Pose Estimation [10.652350454373531]
  We propose a real-time predictive pedestrian collision warning service (P2CWS) for two tasks: pedestrian orientation recognition (100.53 FPS) and intention prediction (35.76 FPS) Our framework obtains satisfying generalization over multiple sites because of the proposed site-independent features. The proposed vision framework realizes 89.3% accuracy in the behavior recognition task on the TUD dataset without any training process.
  arXiv  Detail & Related papers  (2020-09-23T00:55:12Z)
• Deep Learning with Attention Mechanism for Predicting Driver Intention at Intersection [2.1699196439348265]
  The proposed solution is promising to be applied in advanced driver assistance systems (ADAS) and as part of active safety system of autonomous vehicles. The performance of the proposed approach is evaluated on a naturalistic driving dataset and results show that our method achieves high accuracy as well as outperforms other methods.
  arXiv  Detail & Related papers  (2020-06-10T16:12:00Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.