Temporal horizons in forecasting: a performance-learnability trade-off

• URL: http://arxiv.org/abs/2506.03889v2
• Date: Thu, 19 Jun 2025 07:30:55 GMT
• Title: Temporal horizons in forecasting: a performance-learnability trade-off
• Abstract summary: Too short a horizon may miss long-term trends, while too long a horizon can impede convergence due to accumulating prediction errors.<n>We prove that for chaotic systems, the loss landscape's roughness grows exponentially with the training horizon.<n>We also show that models trained on long horizons generalize well to short-term forecasts, whereas those trained on short horizons suffer exponentially (resp. linearly) worse long-term predictions in chaotic (resp. periodic) systems.
• Score: 1.564538938595125
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When training autoregressive models to forecast dynamical systems, a critical question arises: how far into the future should the model be trained to predict? Too short a horizon may miss long-term trends, while too long a horizon can impede convergence due to accumulating prediction errors. In this work, we formalize this trade-off by analyzing how the geometry of the loss landscape depends on the training horizon. We prove that for chaotic systems, the loss landscape's roughness grows exponentially with the training horizon, while for limit cycles, it grows linearly, making long-horizon training inherently challenging. However, we also show that models trained on long horizons generalize well to short-term forecasts, whereas those trained on short horizons suffer exponentially (resp. linearly) worse long-term predictions in chaotic (resp. periodic) systems. We validate our theory through numerical experiments and discuss practical implications for selecting training horizons. Our results provide a principled foundation for hyperparameter optimization in autoregressive forecasting models.

Related papers

• Beyond Myopic World Models: Long-Horizon End-to-End Training for Direct Future Prediction [14.604042842855002]
  We argue that long-horizon accuracy is better achieved by optimizing directly, through an end-to-end endpoint prediction objective.<n>We introduce the Direct Prediction World Model (DPWM), a non-recursive architecture that compresses an action sequence into a single embedding and predicts the endpoint observation in a single forward pass.
  arXiv  Detail & Related papers  (2026-08-07T17:05:44Z)
• Towards Scaling Law Analysis For Spatiotemporal Weather Data [50.25336689856158]
  We extend neural scaling analysis for autoregressive weather forecasting from single-step training loss to long rollouts and per-channel metrics.<n>We find strong cross-channel and cross-horizon heterogeneity: pooled scaling can look favorable while many channels degrade at late leads.<n>We discuss implications for weighted objectives, horizon-aware curricula, and resource allocation across outputs.
  arXiv  Detail & Related papers  (2026-04-06T18:14:53Z)
• An Optimization Method for Autoregressive Time Series Forecasting [4.467218412104476]
  Current time-series forecasting models are primarily based on transformer-style neural networks.<n>We propose a novel training method for time-series forecasting that enforces two key properties.<n>Our method achieves an MSE reduction of more than 10% compared to iTransformer.
  arXiv  Detail & Related papers  (2026-02-02T16:28:00Z)
• EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecasting [34.793750131438934]
  Long-term weather forecasting is critical for socioeconomic planning and disaster preparedness.<n>We present a novel pipeline across pretraining, finetuning and forecasting to enhance long-context modeling while reducing computational overhead.<n>We introduce an Efficient Multi-scale Transformer (EMFormer) to extract multi-scale features through a single convolution in both training and inference.
  arXiv  Detail & Related papers  (2026-02-01T12:36:01Z)
• Optimal Look-back Horizon for Time Series Forecasting in Federated Learning [26.070107882914844]
  This paper presents a principled framework for adaptive horizon selection in federated time series forecasting.<n>We derive a decomposition of the forecasting loss into a Bayesian term, which reflects irreducible uncertainty.<n>We prove that the total forecasting loss is minimized at the smallest horizon where the irreducible loss starts to saturate, while the approximation loss continues to rise.
  arXiv  Detail & Related papers  (2025-11-16T21:46:54Z)
• Scaling Law for Large-Scale Pre-Training Using Chaotic Time Series and Predictability in Financial Time Series [0.0]
  Time series forecasting plays a critical role in decision-making processes across diverse fields.<n>In this study, we propose a methodology for modeling financial time series by generating artificial chaotic time series.<n>We then apply resampling techniques to simulate financial time series data, which we then use as training samples.
  arXiv  Detail & Related papers  (2025-09-05T08:40:13Z)
• Temporal Difference Flows [82.24174052059352]
  Geometric Horizon Models (GHMs) offer a compelling alternative by directly making predictions of future states.<n>Existing methods are negatively affected by bootstrapping predictions at train time and struggle to generate high-quality predictions at long horizons.<n>This paper introduces Temporal Difference Flows (TD-Flow), which leverages the structure of a novel Bellman equation on probability paths alongside flow-matching techniques to learn accurate GHMs at over 5x the horizon length of prior methods.
  arXiv  Detail & Related papers  (2025-03-12T20:30:07Z)
• A probabilistic framework for learning non-intrusive corrections to long-time climate simulations from short-time training data [12.566163525039558]
  We present a strategy for training neural network models to non-intrusively correct under-resolved long-time simulations of chaotic systems. We demonstrate its ability to accurately predict the anisotropic statistics over time horizons more than 30 times longer than the data seen in training.
  arXiv  Detail & Related papers  (2024-08-02T18:34:30Z)
• Loss Shaping Constraints for Long-Term Time Series Forecasting [79.3533114027664]
  We present a Constrained Learning approach for long-term time series forecasting that respects a user-defined upper bound on the loss at each time-step. We propose a practical Primal-Dual algorithm to tackle it, and aims to demonstrate that it exhibits competitive average performance in time series benchmarks, while shaping the errors across the predicted window.
  arXiv  Detail & Related papers  (2024-02-14T18:20:44Z)
• A Novel Hyperdimensional Computing Framework for Online Time Series Forecasting on the Edge [1.4255659581428337]
  offline deep forecasting models fail to adapt effectively to changes in time-series data. Online deep forecasting models are often expensive and have complex training procedures. In this paper, we reframe the online nonlinear time-series forecasting problem as one of linear hyperdimensional time-series forecasting.
  arXiv  Detail & Related papers  (2024-02-03T02:42:53Z)
• ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons [23.9530536685668]
  We present ProbTS, a benchmark tool designed as a unified platform to evaluate fundamental forecasting needs. We dissect the distinctive data characteristics arising from disparate forecasting requirements. We examine the latest models for universal time-series forecasting and discover that our analyses of methodological strengths and weaknesses are also applicable.
  arXiv  Detail & Related papers  (2023-10-11T12:48:45Z)
• Performative Time-Series Forecasting [64.03865043422597]
  We formalize performative time-series forecasting (PeTS) from a machine-learning perspective.<n>We propose a novel approach, Feature Performative-Shifting (FPS), which leverages the concept of delayed response to anticipate distribution shifts.<n>We conduct comprehensive experiments using multiple time-series models on COVID-19 and traffic forecasting tasks.
  arXiv  Detail & Related papers  (2023-10-09T18:34:29Z)
• Distilling Knowledge for Short-to-Long Term Trajectory Prediction [9.626916225081613]
  Long-term trajectory forecasting is an important problem in the fields of computer vision, machine learning, and robotics. We propose Di-Long, a new method that employs the distillation of a short-term trajectory model forecaster that guides a student network for long-term trajectory prediction. Our experiments show that our proposed Di-Long method is effective for long-term forecasting and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-05-15T11:30:28Z)
• WaveBound: Dynamic Error Bounds for Stable Time Series Forecasting [30.692056599222926]
  Time series forecasting has become a critical task due to its high practicality in real-world applications. Recent deep-learning-based approaches have shown remarkable success in time series forecasting. Deep networks still suffer from unstable training and overfitting.
  arXiv  Detail & Related papers  (2022-10-25T19:58:02Z)
• Deep Probabilistic Koopman: Long-term time-series forecasting under periodic uncertainties [7.305019142196582]
  We introduce a surprisingly simple approach that characterizes time-varying distributions and enables reasonably accurate predictions thousands of timesteps into the future. This technique, which we call Deep Probabilistic Koopman (DPK), is based on recent advances in linear Koopman operator theory. We demonstrate the long-term forecasting performance of these models on a diversity of domains, including electricity demand forecasting, atmospheric chemistry, and neuroscience.
  arXiv  Detail & Related papers  (2021-06-10T20:22:41Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting [54.273455592965355]
  Uncertainty in future trajectories stems from two sources: (a) sources known to the agent but unknown to the model, such as long term goals and (b)sources that are unknown to both the agent & the model, such as intent of other agents & irreducible randomness indecisions. We model the epistemic un-certainty through multimodality in long term goals and the aleatoric uncertainty through multimodality in waypoints& paths. To exemplify this dichotomy, we also propose a novel long term trajectory forecasting setting, with prediction horizons upto a minute, an order of magnitude longer than prior works.
  arXiv  Detail & Related papers  (2020-12-02T21:01:29Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.