Time Series Foundation Models: Benchmarking Challenges and Requirements

• URL: http://arxiv.org/abs/2510.13654v1
• Date: Wed, 15 Oct 2025 15:15:45 GMT
• Title: Time Series Foundation Models: Benchmarking Challenges and Requirements
• Authors: Marcel Meyer, Sascha Kaltenpoth, Kevin Zalipski, Oliver Müller,
• Abstract summary: Time Series Foundation Models (TSFMs) represent a new paradigm for time series forecasting.<n> evaluating TSFMs is tricky, as with ever more extensive training sets, it becomes more challenging to ensure integrity benchmarking data.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time Series Foundation Models (TSFMs) represent a new paradigm for time series forecasting, offering zero-shot forecasting capabilities without the need for domain-specific pre-training or fine-tuning. However, as with Large Language Models (LLMs), evaluating TSFMs is tricky, as with ever more extensive training sets, it becomes more and more challenging to ensure the integrity of benchmarking data. Our investigation of existing TSFM evaluation highlights multiple challenges, ranging from the representativeness of the benchmark datasets, over the lack of spatiotemporal evaluation, to risks of information leakage due to overlapping and obscure datasets, and the memorization of global patterns caused by external shocks like economic crises or pandemics. Our findings reveal widespread confusion regarding data partitions, risking inflated performance estimates and incorrect transfer of global knowledge to local time series. We argue for the development of robust evaluation methodologies to prevent pitfalls already observed in LLM and classical time series benchmarking, and call upon the research community to design new, principled approaches, such as evaluations on truly out-of-sample future data, to safeguard the integrity of TSFM assessment.

Related papers

• It's TIME: Towards the Next Generation of Time Series Forecasting Benchmarks [87.7937890373758]
  Time series foundation models (TSFMs) are revolutionizing the forecasting landscape from specific dataset modeling to generalizable task evaluation.<n>We introduce TIME, a next-generation task-centric benchmark comprising 50 fresh datasets and 98 forecasting tasks.<n>We propose a novel pattern-level evaluation perspective that moves beyond traditional dataset-level evaluations based on static meta labels.
  arXiv  Detail & Related papers  (2026-02-12T16:31:01Z)
• TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
  Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
  arXiv  Detail & Related papers  (2026-01-30T17:28:56Z)
• Re(Visiting) Time Series Foundation Models in Finance [3.295157175236371]
  Financial time series forecasting is central to trading, portfolio optimization, and risk management.<n>Recent advances in time series foundation models (TSFMs) offer a new paradigm for learning generalizable temporal representations from large and diverse datasets.<n>This paper presents the first comprehensive empirical study of TSFMs in global financial markets.
  arXiv  Detail & Related papers  (2025-11-23T18:44:19Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Are Time-Series Foundation Models Deployment-Ready? A Systematic Study of Adversarial Robustness Across Domains [23.9530536685668]
  Time Series Foundation Models (TSFMs) are pretrained on large-scale, cross-domain data and capable of zero-shot forecasting in new scenarios without further training.<n>Are TSFMs robust to adversarial input perturbations?<n>These perturbations could be exploited in man-in-the-middle attacks or data poisoning.
  arXiv  Detail & Related papers  (2025-05-26T01:24:11Z)
• Empowering Time Series Analysis with Synthetic Data: A Survey and Outlook in the Era of Foundation Models [104.17057231661371]
  Time series analysis is crucial for understanding dynamics of complex systems.<n>Recent advances in foundation models have led to task-agnostic Time Series Foundation Models (TSFMs) and Large Language Model-based Time Series Models (TSLLMs)<n>Their success depends on large, diverse, and high-quality datasets, which are challenging to build due to regulatory, diversity, quality, and quantity constraints.<n>This survey provides a comprehensive review of synthetic data for TSFMs and TSLLMs, analyzing data generation strategies, their role in model pretraining, fine-tuning, and evaluation, and identifying future research directions.
  arXiv  Detail & Related papers  (2025-03-14T13:53:46Z)
• TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecaster [14.512119661418522]
  We present TS-RAG, a retrieval-augmented generation framework for time series forecasting.<n>Specifically, TS-RAG leverages pre-trained time series encoders to retrieve semantically relevant segments from a dedicated knowledge base.<n>We show that TS-RAG achieves state-of-the-art zero-shot forecasting performance, outperforming the existing TSFMs by up to 6.84% across diverse domains.
  arXiv  Detail & Related papers  (2025-03-06T16:48:48Z)
• An Adversarial Learning Approach to Irregular Time-Series Forecasting [0.032771631221674334]
  We propose an adversarial learning framework with a deep analysis of adversarial components to better capture the nuances of irregular time series.<n>Overall, this research provides practical insights for improving models and evaluation metrics, and pioneers the application of adversarial learning in the domian of irregular time-series forecasting.
  arXiv  Detail & Related papers  (2024-11-28T19:28:07Z)
• Foundation Models for Time Series Analysis: A Tutorial and Survey [70.43311272903334]
  Foundation Models (FMs) have fundamentally reshaped the paradigm of model design for time series analysis. This survey aims to furnish a comprehensive and up-to-date overview of FMs for time series analysis.
  arXiv  Detail & Related papers  (2024-03-21T10:08:37Z)
• Can LMs Generalize to Future Data? An Empirical Analysis on Text Summarization [50.20034493626049]
  Recent pre-trained language models (PLMs) achieve promising results in existing abstractive summarization datasets. Existing summarization benchmarks overlap in time with the standard pre-training corpora and finetuning datasets. We show that parametric knowledge stored in summarization models significantly affects the faithfulness of the generated summaries on future data.
  arXiv  Detail & Related papers  (2023-05-03T08:08:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.