Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

• URL: http://arxiv.org/abs/2607.23146v1
• Date: Sat, 25 Jul 2026 10:55:06 GMT
• Title: Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting
• Abstract summary: Foundation models have emerged as a promising paradigm for zero-shot time series forecasting.<n>They are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting.<n>This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems.
• Score: 26.28141834580785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for zero-shot time series forecasting, enabling accurate predictions on datasets never seen during pre-training. Ranging from tens to hundreds of millions of parameters, these models are pre-trained on vast and diverse collections of time series, learning generalizable representations that support both point and probabilistic forecasting. This approach alleviates the need for dataset-specific model design and manual tuning, offering a unified solution across forecasting problems. In this work, we review the main architectures, pre-training strategies, and optimization methods underpinning these models. We further investigate post-pre-training fine-tuning of selected foundation models to enhance their performance on specific datasets. Our empirical results demonstrate that this step consistently improves forecasting accuracy over the zero-shot baseline.

Related papers

• Modèles de Fondation et Ajustement : Vers une Nouvelle Génération de Modèles pour la Prévision des Séries Temporelles [26.28141834580785]
  Foundations models have been developed for zero-shot time series forecasting.<n>These models learn generalizable representations for both point and probabilistic forecasting.<n>We study the effect of fine-tuning after pretraining to enhance their performance on specific datasets.
  arXiv  Detail & Related papers  (2025-11-27T18:19:20Z)
• Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification [19.714904955821623]
  We show that the best forecasting models achieve classification accuracy that matches or even surpasses that of state-of-the-art models pre-trained specifically for classification.<n>These findings challenge the assumption that task-specific pre-training is necessary, and suggest that learning to forecast may provide a powerful route toward constructing general-purpose time series foundation models.
  arXiv  Detail & Related papers  (2025-10-30T17:55:23Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• ARIES: Relation Assessment and Model Recommendation for Deep Time Series Forecasting [54.57031153712623]
  ARIES is a framework for assessing relation between time series properties and modeling strategies.<n>We propose the first deep forecasting model recommender, capable of providing interpretable suggestions for real-world time series.
  arXiv  Detail & Related papers  (2025-09-07T13:57:14Z)
• ChronosX: Adapting Pretrained Time Series Models with Exogenous Variables [30.679739751673655]
  This paper introduces a new method to incorporate covariates into pretrained time series forecasting models.<n>Our proposed approach incorporates covariate information into pretrained forecasting models through modular blocks.<n>In evaluations on both synthetic and real datasets, our approach effectively incorporates covariate information into pretrained models, outperforming existing baselines.
  arXiv  Detail & Related papers  (2025-03-15T12:34:19Z)
• Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization [74.3339999119713]
  We develop a wavelet-based tokenizer that allows models to learn complex representations directly in the space of time-localized frequencies.<n>Our method first scales and decomposes the input time series, then thresholds and quantizes the wavelet coefficients, and finally pre-trains an autoregressive model to forecast coefficients for the forecast horizon.
  arXiv  Detail & Related papers  (2024-12-06T18:22:59Z)
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• ReAugment: Model Zoo-Guided RL for Few-Shot Time Series Augmentation and Forecasting [74.00765474305288]
  We present a pilot study on using reinforcement learning (RL) for time series data augmentation.<n>Our method, ReAugment, tackles three critical questions: which parts of the training set should be augmented, how the augmentation should be performed, and what advantages RL brings to the process.
  arXiv  Detail & Related papers  (2024-09-10T07:34:19Z)
• Predictive Churn with the Set of Good Models [61.00058053669447]
  This paper explores connections between two seemingly unrelated concepts of predictive inconsistency.<n>The first, known as predictive multiplicity, occurs when models that perform similarly produce conflicting predictions for individual samples.<n>The second concept, predictive churn, examines the differences in individual predictions before and after model updates.
  arXiv  Detail & Related papers  (2024-02-12T16:15:25Z)
• Deep Non-Parametric Time Series Forecaster [19.800783133682955]
  The proposed approach does not assume any parametric form for the predictive distribution and instead generates predictions by sampling from the empirical distribution according to a tunable strategy. We develop a global version of the proposed method that automatically learns the sampling strategy by exploiting the information across multiple related time series.
  arXiv  Detail & Related papers  (2023-12-22T12:46:30Z)
• A decoder-only foundation model for time-series forecasting [23.824504640087753]
  Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus. It can work well across different forecasting history lengths, prediction lengths and temporal granularities.
  arXiv  Detail & Related papers  (2023-10-14T17:01:37Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Investigating Ensemble Methods for Model Robustness Improvement of Text Classifiers [66.36045164286854]
  We analyze a set of existing bias features and demonstrate there is no single model that works best for all the cases. By choosing an appropriate bias model, we can obtain a better robustness result than baselines with a more sophisticated model design.
  arXiv  Detail & Related papers  (2022-10-28T17:52:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.