Closing the Loop: Motion Prediction Models beyond Open-Loop Benchmarks

• URL: http://arxiv.org/abs/2505.05638v1
• Date: Thu, 08 May 2025 20:38:49 GMT
• Title: Closing the Loop: Motion Prediction Models beyond Open-Loop Benchmarks
• Authors: Mohamed-Khalil Bouzidi, Christian Schlauch, Nicole Scheuerer, Yue Yao, Nadja Klein, Daniel Göhring, Jörg Reichardt,
• Abstract summary: We evaluate the interplay between state-of-the-art motion predictors and motion planners.<n>Our results show that higher open-loop accuracy does not always correlate with better closed-loop driving behavior.<n>In some cases models with up to 86% fewer parameters yield comparable or even superior closed-loop driving performance.
• Score: 2.17300236125078
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fueled by motion prediction competitions and benchmarks, recent years have seen the emergence of increasingly large learning based prediction models, many with millions of parameters, focused on improving open-loop prediction accuracy by mere centimeters. However, these benchmarks fail to assess whether such improvements translate to better performance when integrated into an autonomous driving stack. In this work, we systematically evaluate the interplay between state-of-the-art motion predictors and motion planners. Our results show that higher open-loop accuracy does not always correlate with better closed-loop driving behavior and that other factors, such as temporal consistency of predictions and planner compatibility, also play a critical role. Furthermore, we investigate downsized variants of these models, and, surprisingly, find that in some cases models with up to 86% fewer parameters yield comparable or even superior closed-loop driving performance. Our code is available at https://github.com/continental/pred2plan.

Related papers

• Improving Consistency in Vehicle Trajectory Prediction Through Preference Optimization [4.506411269983418]
  Trajectory prediction is an essential step in the pipeline of an autonomous vehicle.<n>Current deep-learning-based trajectory prediction models can achieve excellent accuracy on public datasets.<n>This work fine-tunes trajectory prediction models in multi-agent settings using preference optimization.
  arXiv  Detail & Related papers  (2025-07-03T07:59:49Z)
• Pseudo-Simulation for Autonomous Driving [54.0732376977553]
  Existing evaluation paradigms for Autonomous Vehicles (AVs) face critical limitations.<n>Real-world evaluation is often challenging due to safety concerns and a lack of realism.<n>Open-loop evaluation relies on metrics that generally overlook compounding errors.
  arXiv  Detail & Related papers  (2025-06-04T17:57:53Z)
• Planning with Adaptive World Models for Autonomous Driving [50.4439896514353]
  We present nuPlan, a real-world motion planning benchmark that captures multi-agent interactions.<n>We learn to model such unique behaviors with BehaviorNet, a graph convolutional neural network (GCNN)<n>We also present AdaptiveDriver, a model-predictive control (MPC) based planner that unrolls different world models conditioned on BehaviorNet's predictions.
  arXiv  Detail & Related papers  (2024-06-15T18:53:45Z)
• Valeo4Cast: A Modular Approach to End-to-End Forecasting [93.86257326005726]
  Our solution ranks first in the Argoverse 2 End-to-end Forecasting Challenge, with 63.82 mAPf. We depart from the current trend of tackling this task via end-to-end training from perception to forecasting, and instead use a modular approach. We surpass forecasting results by +17.1 points over last year's winner and by +13.3 points over this year's runner-up.
  arXiv  Detail & Related papers  (2024-06-12T11:50:51Z)
• GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Model [18.762609012554147]
  Trajectory prediction has been a long-standing problem in intelligent systems like autonomous driving and robot navigation. This paper proposes an attention-based graph model, named GATraj, which achieves a good balance of prediction accuracy and inference speed.
  arXiv  Detail & Related papers  (2022-09-16T11:29:19Z)
• Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction [13.105275905781632]
  Pedestrian trajectory prediction is a key technology in many applications such as video surveillance, social robot navigation, and autonomous driving. This work proposes a novel trajectory prediction method called CSR, which consists of a cascaded conditional autoencoder (CVAE) module and a socially-aware regression module. Experiments results demonstrate that the proposed method exhibits improvements over state-of-the-art method on the Stanford Drone dataset.
  arXiv  Detail & Related papers  (2021-10-28T10:56:21Z)
• You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction [52.442129609979794]
  Recent deep learning approaches for trajectory prediction show promising performance. It remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions of different cues to model performance.
  arXiv  Detail & Related papers  (2021-10-11T14:24:15Z)
• Motion Prediction Using Temporal Inception Module [96.76721173517895]
  We propose a Temporal Inception Module (TIM) to encode human motion. Our framework produces input embeddings using convolutional layers, by using different kernel sizes for different input lengths. The experimental results on standard motion prediction benchmark datasets Human3.6M and CMU motion capture dataset show that our approach consistently outperforms the state of the art methods.
  arXiv  Detail & Related papers  (2020-10-06T20:26:01Z)
• AutoCP: Automated Pipelines for Accurate Prediction Intervals [84.16181066107984]
  This paper proposes an AutoML framework called Automatic Machine Learning for Conformal Prediction (AutoCP) Unlike the familiar AutoML frameworks that attempt to select the best prediction model, AutoCP constructs prediction intervals that achieve the user-specified target coverage rate. We tested AutoCP on a variety of datasets and found that it significantly outperforms benchmark algorithms.
  arXiv  Detail & Related papers  (2020-06-24T23:13:11Z)
• Long-Short Term Spatiotemporal Tensor Prediction for Passenger Flow Profile [15.875569404476495]
  We focus on a tensor-based prediction and propose several practical techniques to improve prediction. For long-term prediction specifically, we propose the "Tensor Decomposition + 2-Dimensional Auto-Regressive Moving Average (2D-ARMA)" model. For short-term prediction, we propose to conduct tensor completion based on tensor clustering to avoid oversimplifying and ensure accuracy.
  arXiv  Detail & Related papers  (2020-04-23T08:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.