ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons

• URL: http://arxiv.org/abs/2310.07446v5
• Date: Mon, 21 Oct 2024 11:45:04 GMT
• Title: ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons
• Authors: Jiawen Zhang, Xumeng Wen, Zhenwei Zhang, Shun Zheng, Jia Li, Jiang Bian,
• Abstract summary: We present ProbTS, a benchmark tool designed as a unified platform to evaluate fundamental forecasting needs. We dissect the distinctive data characteristics arising from disparate forecasting requirements. We examine the latest models for universal time-series forecasting and discover that our analyses of methodological strengths and weaknesses are also applicable.
• Score: 23.9530536685668
• License:
• Abstract: Delivering precise point and distributional forecasts across a spectrum of prediction horizons represents a significant and enduring challenge in the application of time-series forecasting within various industries. Prior research on developing deep learning models for time-series forecasting has often concentrated on isolated aspects, such as long-term point forecasting or short-term probabilistic estimations. This narrow focus may result in skewed methodological choices and hinder the adaptability of these models to uncharted scenarios. While there is a rising trend in developing universal forecasting models, a thorough understanding of their advantages and drawbacks, especially regarding essential forecasting needs like point and distributional forecasts across short and long horizons, is still lacking. In this paper, we present ProbTS, a benchmark tool designed as a unified platform to evaluate these fundamental forecasting needs and to conduct a rigorous comparative analysis of numerous cutting-edge studies from recent years. We dissect the distinctive data characteristics arising from disparate forecasting requirements and elucidate how these characteristics can skew methodological preferences in typical research trajectories, which often fail to fully accommodate essential forecasting needs. Building on this, we examine the latest models for universal time-series forecasting and discover that our analyses of methodological strengths and weaknesses are also applicable to these universal models. Finally, we outline the limitations inherent in current research and underscore several avenues for future exploration.

Related papers

• Forecasting with Deep Learning: Beyond Average of Average of Average Performance [0.393259574660092]
  Current practices for evaluating and comparing forecasting models focus on summarising performance into a single score. We propose a novel framework for evaluating models from multiple perspectives. We show the advantages of this framework by comparing a state-of-the-art deep learning approach with classical forecasting techniques.
  arXiv  Detail & Related papers  (2024-06-24T12:28:22Z)
• Deep learning for precipitation nowcasting: A survey from the perspective of time series forecasting [4.5424061912112474]
  This paper reviews recent progress in time series precipitation forecasting models using deep learning. We categorize forecasting models into textitrecursive and textitmultiple strategies based on their approaches to predict future frames. We evaluate current deep learning-based models for precipitation forecasting on a public benchmark, discuss their limitations and challenges, and present some promising research directions.
  arXiv  Detail & Related papers  (2024-06-07T12:07:09Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2023-10-17T20:30:16Z)
• Performative Time-Series Forecasting [71.18553214204978]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective. We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts. We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts [73.33395097728128]
  We provide the first systematic finite-sample study of proper scoring rules for time-series forecasting evaluation. We carry out our analysis on a comprehensive synthetic benchmark, specifically designed to test several key discrepancies between ground-truth and forecast distributions.
  arXiv  Detail & Related papers  (2023-04-19T17:38:42Z)
• When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting [69.30930115236228]
  Probabilistic hierarchical time-series forecasting is an important variant of time-series forecasting. Most methods focus on point predictions and do not provide well-calibrated probabilistic forecasts distributions. We propose PROFHiT, a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
  arXiv  Detail & Related papers  (2022-06-16T06:13:53Z)
• LoMEF: A Framework to Produce Local Explanations for Global Model Time Series Forecasts [2.3096751699592137]
  Global Forecasting Models (GFM) that are trained across a set of multiple time series have shown superior results in many forecasting competitions and real-world applications. However, GFMs typically lack interpretability, especially towards particular time series. We propose a novel local model-agnostic interpretability approach to explain the forecasts from GFMs.
  arXiv  Detail & Related papers  (2021-11-13T00:17:52Z)
• RNN with Particle Flow for Probabilistic Spatio-temporal Forecasting [30.277213545837924]
  Many classical statistical models often fall short in handling the complexity and high non-linearity present in time-series data. In this work, we consider the time-series data as a random realization from a nonlinear state-space model. We use particle flow as the tool for approximating the posterior distribution of the states, as it is shown to be highly effective in complex, high-dimensional settings.
  arXiv  Detail & Related papers  (2021-06-10T21:49:23Z)
• Deep Probabilistic Koopman: Long-term time-series forecasting under periodic uncertainties [7.305019142196582]
  We introduce a surprisingly simple approach that characterizes time-varying distributions and enables reasonably accurate predictions thousands of timesteps into the future. This technique, which we call Deep Probabilistic Koopman (DPK), is based on recent advances in linear Koopman operator theory. We demonstrate the long-term forecasting performance of these models on a diversity of domains, including electricity demand forecasting, atmospheric chemistry, and neuroscience.
  arXiv  Detail & Related papers  (2021-06-10T20:22:41Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting [54.273455592965355]
  Uncertainty in future trajectories stems from two sources: (a) sources known to the agent but unknown to the model, such as long term goals and (b)sources that are unknown to both the agent & the model, such as intent of other agents & irreducible randomness indecisions. We model the epistemic un-certainty through multimodality in long term goals and the aleatoric uncertainty through multimodality in waypoints& paths. To exemplify this dichotomy, we also propose a novel long term trajectory forecasting setting, with prediction horizons upto a minute, an order of magnitude longer than prior works.
  arXiv  Detail & Related papers  (2020-12-02T21:01:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.