Seeking SOTA: Time-Series Forecasting Must Adopt Taxonomy-Specific Evaluation to Dispel Illusory Gains

• URL: http://arxiv.org/abs/2603.15506v1
• Date: Mon, 16 Mar 2026 16:34:26 GMT
• Title: Seeking SOTA: Time-Series Forecasting Must Adopt Taxonomy-Specific Evaluation to Dispel Illusory Gains
• Abstract summary: We argue that the current practice of evaluating AI/ML time-series forecasting models obscures real progress by overlooking the performance of efficient classical methods.<n>We demonstrate that these "standard" datasets often exhibit dominant autocorrelation patterns and seasonal cycles that can be effectively captured by simpler linear or statistical models.
• Score: 17.650604674194984
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We argue that the current practice of evaluating AI/ML time-series forecasting models, predominantly on benchmarks characterized by strong, persistent periodicities and seasonalities, obscures real progress by overlooking the performance of efficient classical methods. We demonstrate that these "standard" datasets often exhibit dominant autocorrelation patterns and seasonal cycles that can be effectively captured by simpler linear or statistical models, rendering complex deep learning architectures frequently no more performant than their classical counterparts for these specific data characteristics, and raising questions as to whether any marginal improvements justify the significant increase in computational overhead and model complexity. We call on the community to (I) retire or substantially augment current benchmarks with datasets exhibiting a wider spectrum of non-stationarities, such as structural breaks, time-varying volatility, and concept drift, and less predictable dynamics drawn from diverse real-world domains, and (II) require every deep learning submission to include robust classical and simple baselines, appropriately chosen for the specific characteristics of the downstream tasks' time series. By doing so, we will help ensure that reported gains reflect genuine scientific methodological advances rather than artifacts of benchmark selection favoring models adept at learning repetitive patterns.

Related papers

• Attention-based Experience Replay Framework for Continual Learning of Agnostic Time Series Forecasting Models [0.9668407688201359]
  Continual learning aims to develop models capable of adapting incrementally while maintaining a balance between stability and plasticity under computational constraints.<n>We introduce a novel framework for continual time series forecasting, designed to extend existing static forecasting models.<n>Results show that our approach effectively increases or maintains predictive performance over time while reducing retraining costs and data requirements.
  arXiv  Detail & Related papers  (2026-06-12T11:44:05Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices [0.0]
  We present an enhanced StockBot architecture that systematically evaluates modern attention-based, convolutional, and recurrent time-series forecasting models.<n>A carefully constructed vanilla LSTM consistently achieves superior predictive accuracy and more stable buy/sell decision-making.
  arXiv  Detail & Related papers  (2026-01-01T04:09:51Z)
• Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data [89.96277093034547]
  We introduce EntroDrop, an entropy-guided token dropout method that functions as structured data regularization.<n>We show that EntroDrop consistently outperforms standard regularization baselines and maintains robust performance throughout extended multi-epoch training.
  arXiv  Detail & Related papers  (2025-12-29T12:35:51Z)
• SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series [11.314952720053464]
  We propose a synthetic data-driven evaluation paradigm, SynTSBench, for time series forecasting models.<n>Our framework isolates confounding factors and establishes an interpretable evaluation system with three core analytical dimensions.<n>Our experiments show that current deep learning models do not universally approach optimal baselines across all types of temporal features.
  arXiv  Detail & Related papers  (2025-10-23T06:59:38Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• Estimating Time Series Foundation Model Transferability via In-Context Learning [74.65355820906355]
  Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training.<n>Fine-tuning remains critical for boosting performance in domains with limited public data.<n>We introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem.
  arXiv  Detail & Related papers  (2025-09-28T07:07:13Z)
• Characteristic Root Analysis and Regularization for Linear Time Series Forecasting [9.254995889539716]
  Time series forecasting remains a critical challenge across numerous domains.<n>Recent studies highlight the surprising competitiveness of simple linear models.<n>This paper focuses on the role of characteristic roots in temporal dynamics.
  arXiv  Detail & Related papers  (2025-09-28T03:06:30Z)
• VARMA-Enhanced Transformer for Time Series Forecasting [4.982130518684668]
  VARMAformer is a novel architecture that synergizes the efficiency of a cross-attention-only framework with the principles of classical time series analysis.<n>By fusing these classical insights into a modern backbone, VARMAformer captures both global, long-range dependencies and local, statistical structures.
  arXiv  Detail & Related papers  (2025-09-05T03:32:51Z)
• Frequency-Constrained Learning for Long-Term Forecasting [15.31488551912888]
  Real-world time series exhibit strong periodic structures arising from physical laws, human routines, or seasonal cycles.<n>Modern deep forecasting models often fail to capture these recurring patterns due to spectral bias and a lack of frequency-aware inductive priors.<n>We propose a simple yet effective method that enhances long-term forecasting by explicitly modeling periodicity.
  arXiv  Detail & Related papers  (2025-08-02T22:12:15Z)
• Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization [74.3339999119713]
  We develop a wavelet-based tokenizer that allows models to learn complex representations directly in the space of time-localized frequencies.<n>Our method first scales and decomposes the input time series, then thresholds and quantizes the wavelet coefficients, and finally pre-trains an autoregressive model to forecast coefficients for the forecast horizon.
  arXiv  Detail & Related papers  (2024-12-06T18:22:59Z)
• Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting [46.63798583414426]
  Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis. Our study demonstrates, through both analytical and empirical evidence, that decomposition is key to containing excessive model inflation. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks.
  arXiv  Detail & Related papers  (2024-01-22T13:15:40Z)
• Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs [50.25683648762602]
  We introduce Koopman VAE, a new generative framework that is based on a novel design for the model prior. Inspired by Koopman theory, we represent the latent conditional prior dynamics using a linear map. KoVAE outperforms state-of-the-art GAN and VAE methods across several challenging synthetic and real-world time series generation benchmarks.
  arXiv  Detail & Related papers  (2023-10-04T07:14:43Z)
• OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning [67.07363529640784]
  We propose OpenSTL to categorize prevalent approaches into recurrent-based and recurrent-free models. We conduct standard evaluations on datasets across various domains, including synthetic moving object trajectory, human motion, driving scenes, traffic flow and forecasting weather. We find that recurrent-free models achieve a good balance between efficiency and performance than recurrent models.
  arXiv  Detail & Related papers  (2023-06-20T03:02:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.