QuitoBench: A High-Quality Open Time Series Forecasting Benchmark

• URL: http://arxiv.org/abs/2603.26017v1
• Date: Fri, 27 Mar 2026 02:24:34 GMT
• Title: QuitoBench: A High-Quality Open Time Series Forecasting Benchmark
• Abstract summary: Time series forecasting is critical across finance, healthcare, and cloud computing.<n>We introduce textscQuitoBench, a regime-balanced benchmark for time series forecasting.
• Score: 32.21290355342465
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Time series forecasting is critical across finance, healthcare, and cloud computing, yet progress is constrained by a fundamental bottleneck: the scarcity of large-scale, high-quality benchmarks. To address this gap, we introduce \textsc{QuitoBench}, a regime-balanced benchmark for time series forecasting with coverage across eight trend$\times$seasonality$\times$forecastability (TSF) regimes, designed to capture forecasting-relevant properties rather than application-defined domain labels. The benchmark is built upon \textsc{Quito}, a billion-scale time series corpus of application traffic from Alipay spanning nine business domains. Benchmarking 10 models from deep learning, foundation models, and statistical baselines across 232,200 evaluation instances, we report four key findings: (i) a context-length crossover where deep learning models lead at short context ($L=96$) but foundation models dominate at long context ($L \ge 576$); (ii) forecastability is the dominant difficulty driver, producing a $3.64 \times$ MAE gap across regimes; (iii) deep learning models match or surpass foundation models at $59 \times$ fewer parameters; and (iv) scaling the amount of training data provides substantially greater benefit than scaling model size for both model families. These findings are validated by strong cross-benchmark and cross-metric consistency. Our open-source release enables reproducible, regime-aware evaluation for time series forecasting research.

Related papers

• FinStressTS: A Parametric Synthetic Benchmark for Time-Series Forecasting in Finance [3.5492452247462185]
  FinStressTS is a mechanism-aware benchmark that links model behavior to controlled structural causes.<n>Financial forecasting is difficult due to low signal-to-noise ratios, latent factors, heavy tails, regime shifts, and jumps.
  arXiv  Detail & Related papers  (2026-06-02T05:41:50Z)
• Multivariate Financial Forecasting using the Chronos Time Series Foundation Models [0.0]
  Using Chronos-2, we evaluate pretrained time-series models for economic and financial forecasting.<n>Across datasets, MV forecasts consistently outperform UV forecasts.<n>Mixing time series across equity and interest rate markets reduces forecast accuracy.
  arXiv  Detail & Related papers  (2026-05-08T21:04:19Z)
• $V_{0.5}$: Generalist Value Model as a Prior for Sparse RL Rollouts [81.48669089692189]
  Generalist Value Models (such as $V_0.5$) achieve pre-trained value estimation by explicitly encoding model capabilities in-context.<n>We propose $V_0.5$, which adaptively fuses the baseline predicted by such value model with the empirical mean derived from sparse rollouts.<n>$V_0.5$ significantly outperforms GRPO and DAPO, achieving faster convergence and over some 10% performance improvement.
  arXiv  Detail & Related papers  (2026-03-11T14:57:41Z)
• Accuracy Law for the Future of Deep Time Series Forecasting [65.46625911002202]
  Time series forecasting inherently faces a non-zero error lower bound due to its partially observable and uncertain nature.<n>This paper focuses on a fundamental question: how to estimate the performance upper bound of deep time series forecasting.<n>Based on rigorous statistical tests of over 2,800 newly trained deep forecasters, we discover a significant exponential relationship between the minimum forecasting error of deep models and the complexity of window-wise series patterns.
  arXiv  Detail & Related papers  (2025-10-03T05:18:47Z)
• TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models [105.47481207029047]
  We introduce Time Series Reasoning Suite (TSR-Suite), which formalizes four atomic tasks that span three fundamental capabilities for reasoning with time series.<n>We also introduce Time Omni-1, the first unified reasoning model designed to address diverse real-world problems demanding time series reasoning.
  arXiv  Detail & Related papers  (2025-09-29T13:54:34Z)
• Enhancing Transformer-Based Foundation Models for Time Series Forecasting via Bagging, Boosting and Statistical Ensembles [7.787518725874443]
  Time series foundation models (TSFMs) have shown strong generalization and zero-shot capabilities for time series forecasting, anomaly detection, classification, and imputation.<n>This paper investigates a suite of statistical and ensemble-based enhancement techniques to improve robustness and accuracy.
  arXiv  Detail & Related papers  (2025-08-18T04:06:26Z)
• Evaluation of a Foundational Model and Stochastic Models for Forecasting Sporadic or Spiky Production Outages of High-Performance Machine Learning Services [0.0]
  We optimize a state-of-the-art foundational model to forecast sporadic or spiky production outages of high-performance machine learning services.<n>The analysis helps us understand how each of the evaluated models performs for the sporadic or spiky events.<n>We use the models with optimal parameters to estimate a year-long outage statistics of a particular root cause with less than 6% value errors.
  arXiv  Detail & Related papers  (2025-06-30T23:59:12Z)
• When Does Multimodality Lead to Better Time Series Forecasting? [96.26052272121615]
  We investigate whether and under what conditions such multimodal integration consistently yields gains.<n>Our findings reveal that the benefits of multimodality are highly condition-dependent.<n>Our study offers a rigorous, quantitative foundation for understanding when multimodality can be expected to aid forecasting tasks.
  arXiv  Detail & Related papers  (2025-06-20T23:55:56Z)
• Intention-Conditioned Flow Occupancy Models [80.42634994902858]
  Large-scale pre-training has fundamentally changed how machine learning research is done today.<n>Applying this same framework to reinforcement learning is appealing because it offers compelling avenues for addressing core challenges in RL.<n>Recent advances in generative AI have provided new tools for modeling highly complex distributions.
  arXiv  Detail & Related papers  (2025-06-10T15:27:46Z)
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  Time series foundation models excel in zero-shot forecasting, handling diverse tasks without explicit training. GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 23 datasets over 144,000 time series and 177 million data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• LENS: Large Pre-trained Transformer for Exploring Financial Time Series Regularities [3.475989206546412]
  We propose textbfLENS, a pre-trained model for financial time series.<n>textbfLENS effectively captures the complexity of financial systems through a carefully crafted model architecture.<n>Our work offers practical insights into developing pre-trained time series models in high-noise environments.
  arXiv  Detail & Related papers  (2024-08-19T15:59:46Z)
• A Scalable and Transferable Time Series Prediction Framework for Demand Forecasting [24.06534393565697]
  Time series forecasting is one of the most essential and ubiquitous tasks in many business problems. We propose Forecasting orchestra (Forchestra), a simple but powerful framework capable of accurately predicting future demand for a diverse range of items.
  arXiv  Detail & Related papers  (2024-02-29T18:01:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.