Streaming Motion Forecasting for Autonomous Driving

• URL: http://arxiv.org/abs/2310.01351v1
• Date: Mon, 2 Oct 2023 17:13:16 GMT
• Title: Streaming Motion Forecasting for Autonomous Driving
• Authors: Ziqi Pang, Deva Ramanan, Mengtian Li, Yu-Xiong Wang
• Abstract summary: We introduce a benchmark that queries future trajectories on streaming data and we refer to it as "streaming forecasting" Our benchmark inherently captures the disappearance and re-appearance of agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks. We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can adapt any snapshot-based forecaster into a streaming forecaster.
• Score: 71.7468645504988
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Trajectory forecasting is a widely-studied problem for autonomous navigation. However, existing benchmarks evaluate forecasting based on independent snapshots of trajectories, which are not representative of real-world applications that operate on a continuous stream of data. To bridge this gap, we introduce a benchmark that continuously queries future trajectories on streaming data and we refer to it as "streaming forecasting." Our benchmark inherently captures the disappearance and re-appearance of agents, presenting the emergent challenge of forecasting for occluded agents, which is a safety-critical problem yet overlooked by snapshot-based benchmarks. Moreover, forecasting in the context of continuous timestamps naturally asks for temporal coherence between predictions from adjacent timestamps. Based on this benchmark, we further provide solutions and analysis for streaming forecasting. We propose a plug-and-play meta-algorithm called "Predictive Streamer" that can adapt any snapshot-based forecaster into a streaming forecaster. Our algorithm estimates the states of occluded agents by propagating their positions with multi-modal trajectories, and leverages differentiable filters to ensure temporal consistency. Both occlusion reasoning and temporal coherence strategies significantly improve forecasting quality, resulting in 25% smaller endpoint errors for occluded agents and 10-20% smaller fluctuations of trajectories. Our work is intended to generate interest within the community by highlighting the importance of addressing motion forecasting in its intrinsic streaming setting. Code is available at https://github.com/ziqipang/StreamingForecasting.

Related papers

• Multi-modal Motion Prediction using Temporal Ensembling with Learning-based Aggregation [12.716238214021317]
  This paper introduces Temporal Ensembling with Learning-based Aggregation, a meta-algorithm designed to mitigate the issue of missing behaviors in trajectory prediction. By confirming predictions from multiple frames, temporal ensembling compensates for occasional errors in individual frame predictions. Our method, validated on the Argoverse 2 dataset, shows notable improvements.
  arXiv  Detail & Related papers  (2024-10-25T14:59:07Z)
• Scaling transformer neural networks for skillful and reliable medium-range weather forecasting [23.249955524044392]
  We introduce Stormer, a state-of-the-art performance on weather forecasting with minimal changes to the standard transformer backbone. At the core of Stormer is a randomized forecasting objective that trains the model to forecast the weather dynamics over varying time intervals. On WeatherBench 2, Stormer performs competitively at short to medium-range forecasts and outperforms current methods beyond 7 days.
  arXiv  Detail & Related papers  (2023-12-06T19:46:06Z)
• Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive? [93.10694819127608]
  We propose a unified evaluation pipeline for forecasting methods with real-world perception inputs. Our in-depth study uncovers a substantial performance gap when transitioning from curated to perception-based data.
  arXiv  Detail & Related papers  (2023-06-15T17:03:14Z)
• Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty [125.41260574344933]
  We consider an encoder-decoder based deep ensemble network for capturing both perception and predictive uncertainty simultaneously. Overall, deep ensembles provided more robust predictions and the consideration of upstream uncertainty further increased the estimation accuracy for the model.
  arXiv  Detail & Related papers  (2023-05-26T04:27:48Z)
• Practical Adversarial Attacks on Spatiotemporal Traffic Forecasting Models [9.885060319609831]
  Existing methods assume a reliable and unbiased forecasting environment, which is not always available in the wild. We propose a practical adversarial attack framework, instead of simultaneously attacking all data sources. We theoretically demonstrate the worst performance bound of adversarial traffic forecasting attacks.
  arXiv  Detail & Related papers  (2022-10-05T02:25:10Z)
• Real-time Object Detection for Streaming Perception [84.2559631820007]
  Streaming perception is proposed to jointly evaluate the latency and accuracy into a single metric for video online perception. We build a simple and effective framework for streaming perception. Our method achieves competitive performance on Argoverse-HD dataset and improves the AP by 4.9% compared to the strong baseline.
  arXiv  Detail & Related papers  (2022-03-23T11:33:27Z)
• Comparing Sequential Forecasters [35.38264087676121]
  Consider two forecasters, each making a single prediction for a sequence of events over time. How might we compare these forecasters, either online or post-hoc, while avoiding unverifiable assumptions on how the forecasts and outcomes were generated? We present novel sequential inference procedures for estimating the time-varying difference in forecast scores. We empirically validate our approaches by comparing real-world baseball and weather forecasters.
  arXiv  Detail & Related papers  (2021-09-30T22:54:46Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• Towards Streaming Perception [70.68520310095155]
  We present an approach that coherently integrates latency and accuracy into a single metric for real-time online perception. The key insight behind this metric is to jointly evaluate the output of the entire perception stack at every time instant. We focus on the illustrative tasks of object detection and instance segmentation in urban video streams, and contribute a novel dataset with high-quality and temporally-dense annotations.
  arXiv  Detail & Related papers  (2020-05-21T01:51:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.