How Foundational are Foundation Models for Time Series Forecasting?

• URL: http://arxiv.org/abs/2510.00742v3
• Date: Tue, 07 Oct 2025 13:03:30 GMT
• Title: How Foundational are Foundation Models for Time Series Forecasting?
• Authors: Nouha Karaouli, Denis Coquenet, Elisa Fromont, Martial Mermillod, Marina Reyboz,
• Abstract summary: We argue that the inherent diversity of time series data makes foundation models less suited for building effective models.<n>We show that the zero-shot capabilities of a time series foundation model are significantly influenced and tied to the specific domains it has been pretrained on.
• Score: 2.692427265051276
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Foundation Models are designed to serve as versatile embedding machines, with strong zero shot capabilities and superior generalization performance when fine-tuned on diverse downstream tasks. While this is largely true for language and vision foundation models, we argue that the inherent diversity of time series data makes them less suited for building effective foundation models. We demonstrate this using forecasting as our downstream task. We show that the zero-shot capabilities of a time series foundation model are significantly influenced and tied to the specific domains it has been pretrained on. Furthermore, when applied to unseen real-world time series data, fine-tuned foundation models do not consistently yield substantially better results, relative to their increased parameter count and memory footprint, than smaller, dedicated models tailored to the specific forecasting task at hand.

Related papers

• Reverso: Efficient Time Series Foundation Models for Zero-shot Forecasting [38.81240885985943]
  We show that small hybrid models that interleave long convolution and linear RNN layers can match the performance of larger transformer-based models.<n>This recipe results in Reverso, a family of efficient time series foundation models for zero-shot forecasting.
  arXiv  Detail & Related papers  (2026-02-19T18:48:08Z)
• Modèles de Fondation et Ajustement : Vers une Nouvelle Génération de Modèles pour la Prévision des Séries Temporelles [26.28141834580785]
  Foundations models have been developed for zero-shot time series forecasting.<n>These models learn generalizable representations for both point and probabilistic forecasting.<n>We study the effect of fine-tuning after pretraining to enhance their performance on specific datasets.
  arXiv  Detail & Related papers  (2025-11-27T18:19:20Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• ARIES: Relation Assessment and Model Recommendation for Deep Time Series Forecasting [54.57031153712623]
  ARIES is a framework for assessing relation between time series properties and modeling strategies.<n>We propose the first deep forecasting model recommender, capable of providing interpretable suggestions for real-world time series.
  arXiv  Detail & Related papers  (2025-09-07T13:57:14Z)
• Evaluation of a Foundational Model and Stochastic Models for Forecasting Sporadic or Spiky Production Outages of High-Performance Machine Learning Services [0.0]
  We optimize a state-of-the-art foundational model to forecast sporadic or spiky production outages of high-performance machine learning services.<n>The analysis helps us understand how each of the evaluated models performs for the sporadic or spiky events.<n>We use the models with optimal parameters to estimate a year-long outage statistics of a particular root cause with less than 6% value errors.
  arXiv  Detail & Related papers  (2025-06-30T23:59:12Z)
• Output Scaling: YingLong-Delayed Chain of Thought in a Large Pretrained Time Series Forecasting Model [55.25659103706409]
  This framework achieves state-of-the-art performance for our designed foundation model, YingLong.<n>YingLong is a non-causal, bidirectional attention encoder-only transformer trained through masked token recovery.<n>We release four foundation models ranging from 6M to 300M parameters, demonstrating superior results in zero-shot tasks.
  arXiv  Detail & Related papers  (2025-05-20T14:31:06Z)
• Mantis: Lightweight Calibrated Foundation Model for User-Friendly Time Series Classification [16.738168952631735]
  We present Mantis, a new open-source foundation model for time series classification based on the Vision Transformer architecture.<n>Our experimental results show that Mantis outperforms existing foundation models both when the backbone is frozen and when fine-tuned.
  arXiv  Detail & Related papers  (2025-02-21T18:06:09Z)
• Measuring Pre-training Data Quality without Labels for Time Series Foundation Models [10.64362760848387]
  We introduce contrastive accuracy, a new measure to evaluate the quality of the representation space learned by the foundation model.<n>Our experiments reveal the positive correlation between the proposed measure and the accuracy of the model on a collection of downstream tasks.
  arXiv  Detail & Related papers  (2024-12-09T10:38:30Z)
• In-Context Fine-Tuning for Time-Series Foundation Models [18.348874079298298]
  In particular, we design a pretrained foundation model that can be prompted with multiple time-series examples. Our foundation model is specifically trained to utilize examples from multiple related time-series in its context window. We show that such a foundation model that uses in-context examples at inference time can obtain much better performance on popular forecasting benchmarks.
  arXiv  Detail & Related papers  (2024-10-31T16:20:04Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
  This paper aims at the early development of large time series models (LTSM) During pre-training, we curate large-scale datasets with up to 1 billion time points. To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
  arXiv  Detail & Related papers  (2024-02-04T06:55:55Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.