Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios

• URL: http://arxiv.org/abs/2211.05455v1
• Date: Thu, 10 Nov 2022 09:59:38 GMT
• Title: Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios
• Authors: Julian Frederik Schumann, Jens Kober, Arkady Zgonnikov
• Abstract summary: We develop a framework facilitating the evaluation of any model, by any metric, and in any scenario. We then apply this framework to state-of-the-art prediction models, which all show themselves to be unreliable in the most safety-critical situations.
• Score: 4.801975818473341
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Autonomous vehicles currently suffer from a time-inefficient driving style caused by uncertainty about human behavior in traffic interactions. Accurate and reliable prediction models enabling more efficient trajectory planning could make autonomous vehicles more assertive in such interactions. However, the evaluation of such models is commonly oversimplistic, ignoring the asymmetric importance of prediction errors and the heterogeneity of the datasets used for testing. We examine the potential of recasting interactions between vehicles as gap acceptance scenarios and evaluating models in this structured environment. To that end, we develop a framework facilitating the evaluation of any model, by any metric, and in any scenario. We then apply this framework to state-of-the-art prediction models, which all show themselves to be unreliable in the most safety-critical situations.

Related papers

• Diffusion-Based Environment-Aware Trajectory Prediction [3.1406146587437904]
  The ability to predict the future trajectories of traffic participants is crucial for the safe and efficient operation of autonomous vehicles. In this paper, a diffusion-based generative model for multi-agent trajectory prediction is proposed. The model is capable of capturing the complex interactions between traffic participants and the environment, accurately learning the multimodal nature of the data.
  arXiv  Detail & Related papers  (2024-03-18T10:35:15Z)
• Towards Generalizable and Interpretable Motion Prediction: A Deep Variational Bayes Approach [54.429396802848224]
  This paper proposes an interpretable generative model for motion prediction with robust generalizability to out-of-distribution cases. For interpretability, the model achieves the target-driven motion prediction by estimating the spatial distribution of long-term destinations. Experiments on motion prediction datasets validate that the fitted model can be interpretable and generalizable.
  arXiv  Detail & Related papers  (2024-03-10T04:16:04Z)
• Predictive Churn with the Set of Good Models [64.05949860750235]
  We study the effect of conflicting predictions over the set of near-optimal machine learning models. We present theoretical results on the expected churn between models within the Rashomon set. We show how our approach can be used to better anticipate, reduce, and avoid churn in consumer-facing applications.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds [79.00975648564483]
  Trajectory forecasting models, employed in fields such as robotics, autonomous vehicles, and navigation, face challenges in real-world scenarios. This dataset provides comprehensive data, including the locations of all agents, scene images, and point clouds, all from the robot's perspective. The objective is to predict the future positions of agents relative to the robot using raw sensory input data.
  arXiv  Detail & Related papers  (2023-11-05T18:59:31Z)
• Evaluating Pedestrian Trajectory Prediction Methods with Respect to Autonomous Driving [0.9217021281095907]
  In this paper, we assess the state of the art in pedestrian trajectory prediction within the context of generating single trajectories. The evaluation is conducted on the widely-used ETH/UCY dataset where the Average Displacement Error (ADE) and the Final Displacement Error (FDE) are reported.
  arXiv  Detail & Related papers  (2023-08-09T19:21:50Z)
• Using Models Based on Cognitive Theory to Predict Human Behavior in Traffic: A Case Study [4.705182901389292]
  We investigate the usefulness of a novel cognitively plausible model for predicting human behavior in gap acceptance scenarios. We show that this model can compete with or even outperform well-established data-driven prediction models.
  arXiv  Detail & Related papers  (2023-05-24T14:27:00Z)
• Are Neural Topic Models Broken? [81.15470302729638]
  We study the relationship between automated and human evaluation of topic models. We find that neural topic models fare worse in both respects compared to an established classical method.
  arXiv  Detail & Related papers  (2022-10-28T14:38:50Z)
• Scenario-based Evaluation of Prediction Models for Automated Vehicles [0.0]
  We argue that following evaluation practices in safety assessment for automated vehicles should be performed in a scenario-based fashion. We categorize trajectories of Open Motion dataset according to the type of movement they capture. Results show that common evaluation methods are insufficient and the assessment should be performed depending on the application in which the model will operate.
  arXiv  Detail & Related papers  (2022-10-11T09:45:43Z)
• Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [137.00426219455116]
  Under dynamic traffic scenarios, planning based on deterministic predictions is not trustworthy. The authors propose to quantify uncertainty during forecasting using approximation which deterministic approaches fail to capture. The effect of dropout weights and long-term prediction on future state uncertainty has been studied.
  arXiv  Detail & Related papers  (2022-05-04T04:23:38Z)
• Causal-based Time Series Domain Generalization for Vehicle Intention Prediction [19.944268567657307]
  Accurately predicting possible behaviors of traffic participants is an essential capability for autonomous vehicles. In this paper, we aim to address the domain generalization problem for vehicle intention prediction tasks. Our proposed method has consistent improvement on prediction accuracy compared to other state-of-the-art domain generalization and behavior prediction methods.
  arXiv  Detail & Related papers  (2021-12-03T18:58:07Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.