Zero-shot forecasting of chaotic systems

• URL: http://arxiv.org/abs/2409.15771v1
• Date: Tue, 24 Sep 2024 05:56:58 GMT
• Title: Zero-shot forecasting of chaotic systems
• Authors: Yuanzhao Zhang, William Gilpin,
• Abstract summary: Foundation models pre-trained on vast amounts of time-series data from diverse domains. We evaluate whether the zero-shot learning paradigm extends to the challenging task of forecasting chaotic systems.
• Score: 6.445605125467573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time-series forecasting is a challenging task that traditionally requires specialized models custom-trained for the specific task at hand. Recently, inspired by the success of large language models, foundation models pre-trained on vast amounts of time-series data from diverse domains have emerged as a promising candidate for general-purpose time-series forecasting. The defining characteristic of these foundation models is their ability to perform zero-shot learning, that is, forecasting a new system from limited context data without explicit re-training or fine-tuning. Here, we evaluate whether the zero-shot learning paradigm extends to the challenging task of forecasting chaotic systems. Across 135 distinct chaotic dynamical systems and $10^8$ timepoints, we find that foundation models produce competitive forecasts compared to custom-trained models (including NBEATS, TiDE, etc.), particularly when training data is limited. Interestingly, even after point forecasts fail, foundation models preserve the geometric and statistical properties of the chaotic attractors, demonstrating a surprisingly strong ability to capture the long-term behavior of chaotic dynamical systems. Our results highlight the promises and pitfalls of foundation models in making zero-shot forecasts of chaotic systems.

Related papers

• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• Implicit Reasoning in Deep Time Series Forecasting [16.750280337155647]
  This work takes an initial step toward assessing the reasoning abilities of deep time series forecasting models. We find that certain linear, patch-based Transformer models generalize effectively in systematically orchestrated out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2024-09-17T02:11:19Z)
• Koopman Ensembles for Probabilistic Time Series Forecasting [6.699751896019971]
  We show that ensembles of independently trained models are highly overconfident and that using a training criterion that explicitly encourages the members to produce predictions with high inter-model variances greatly improves the uncertainty of the ensembles.
  arXiv  Detail & Related papers  (2024-03-11T14:29:56Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Leveraging the structure of dynamical systems for data-driven modeling [111.45324708884813]
  We consider the impact of the training set and its structure on the quality of the long-term prediction. We show how an informed design of the training set, based on invariants of the system and the structure of the underlying attractor, significantly improves the resulting models.
  arXiv  Detail & Related papers  (2021-12-15T20:09:20Z)
• Neural forecasting at scale [8.245069318446415]
  We study the problem of efficiently scaling ensemble-based deep neural networks for time series (TS) forecasting on a large set of time series. Our model addresses the practical limitations of related models, reducing the training time by half and memory requirement by a factor of 5.
  arXiv  Detail & Related papers  (2021-09-20T17:22:40Z)
• Randomized Neural Networks for Forecasting Time Series with Multiple Seasonality [0.0]
  This work contributes to the development of neural forecasting models with novel randomization-based learning methods. A pattern-based representation of time series makes the proposed approach useful for forecasting time series with multiple seasonality.
  arXiv  Detail & Related papers  (2021-07-04T18:39:27Z)
• Simultaneously Reconciled Quantile Forecasting of Hierarchically Related Time Series [11.004159006784977]
  We propose a flexible nonlinear model that optimize quantile regression loss coupled with suitable regularization terms to maintain consistency of forecasts across hierarchies. The theoretical framework introduced herein can be applied to any forecasting model with an underlying differentiable loss function.
  arXiv  Detail & Related papers  (2021-02-25T00:59:01Z)
• Model-Attentive Ensemble Learning for Sequence Modeling [86.4785354333566]
  We present Model-Attentive Ensemble learning for Sequence modeling (MAES) MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions. We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
  arXiv  Detail & Related papers  (2021-02-23T05:23:35Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.