Unified Long-Term Time-Series Forecasting Benchmark

• URL: http://arxiv.org/abs/2309.15946v1
• Date: Wed, 27 Sep 2023 18:59:00 GMT
• Title: Unified Long-Term Time-Series Forecasting Benchmark
• Authors: Jacek Cyranka, Szymon Haponiuk
• Abstract summary: We present a comprehensive dataset designed explicitly for long-term time-series forecasting. We incorporate a collection of datasets obtained from diverse, dynamic systems and real-life records. To determine the most effective model in diverse scenarios, we conduct an extensive benchmarking analysis using classical and state-of-the-art models. Our findings reveal intriguing performance comparisons among these models, highlighting the dataset-dependent nature of model effectiveness.
• Score: 0.6526824510982802
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In order to support the advancement of machine learning methods for predicting time-series data, we present a comprehensive dataset designed explicitly for long-term time-series forecasting. We incorporate a collection of datasets obtained from diverse, dynamic systems and real-life records. Each dataset is standardized by dividing it into training and test trajectories with predetermined lookback lengths. We include trajectories of length up to $2000$ to ensure a reliable evaluation of long-term forecasting capabilities. To determine the most effective model in diverse scenarios, we conduct an extensive benchmarking analysis using classical and state-of-the-art models, namely LSTM, DeepAR, NLinear, N-Hits, PatchTST, and LatentODE. Our findings reveal intriguing performance comparisons among these models, highlighting the dataset-dependent nature of model effectiveness. Notably, we introduce a custom latent NLinear model and enhance DeepAR with a curriculum learning phase. Both consistently outperform their vanilla counterparts.

Related papers

• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 28 datasets over 144,000 time series and 177 million data points. We also provide a non-leaking pretraining dataset containing approximately 230 billion data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• Test Time Learning for Time Series Forecasting [1.4605709124065924]
  Test-Time Training (TTT) modules consistently outperform state-of-the-art models, including the Mamba-based TimeMachine. Our results show significant improvements in Mean Squared Error (MSE) and Mean Absolute Error (MAE) This work sets a new benchmark for time-series forecasting and lays the groundwork for future research in scalable, high-performance forecasting models.
  arXiv  Detail & Related papers  (2024-09-21T04:40:08Z)
• Deep Time Series Models: A Comprehensive Survey and Benchmark [74.28364194333447]
  Time series data is of great significance in real-world scenarios. Recent years have witnessed remarkable breakthroughs in the time series community. We release Time Series Library (TSLib) as a fair benchmark of deep time series models for diverse analysis tasks.
  arXiv  Detail & Related papers  (2024-07-18T08:31:55Z)
• Understanding Different Design Choices in Training Large Time Series Models [71.20102277299445]
  Training Large Time Series Models (LTSMs) on heterogeneous time series data poses unique challenges. We propose emphtime series prompt, a novel statistical prompting strategy tailored to time series data. We introduce textttLTSM-bundle, which bundles the best design choices we have identified.
  arXiv  Detail & Related papers  (2024-06-20T07:09:19Z)
• Multi-Scale Dilated Convolution Network for Long-Term Time Series Forecasting [17.132063819650355]
  We propose Multi Scale Dilated Convolution Network (MSDCN) to capture the period and trend characteristics of long time series. We design different convolution blocks with exponentially growing dilations and varying kernel sizes to sample time series data at different scales. To validate the effectiveness of the proposed approach, we conduct experiments on eight challenging long-term time series forecasting benchmark datasets.
  arXiv  Detail & Related papers  (2024-05-09T02:11:01Z)
• Chronos: Learning the Language of Time Series [79.38691251254173]
  Chronos is a framework for pretrained probabilistic time series models. We show that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks.
  arXiv  Detail & Related papers  (2024-03-12T16:53:54Z)
• Generative Pretrained Hierarchical Transformer for Time Series Forecasting [3.739587363053192]
  We propose a novel generative pretrained hierarchical transformer architecture for forecasting, named textbfGPHT. We conduct sufficient experiments on eight datasets with mainstream self-supervised pretraining models and supervised models. The results demonstrated that GPHT surpasses the baseline models across various fine-tuning and zero/few-shot learning settings in the traditional long-term forecasting task.
  arXiv  Detail & Related papers  (2024-02-26T11:54:54Z)
• Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain [54.67888148566323]
  We introduce three large-scale time series forecasting datasets from the cloud operations domain. We show it is a strong zero-shot baseline and benefits from further scaling, both in model and dataset size. Accompanying these datasets and results is a suite of comprehensive benchmark results comparing classical and deep learning baselines to our pre-trained method.
  arXiv  Detail & Related papers  (2023-10-08T08:09:51Z)
• Comparing Test Sets with Item Response Theory [53.755064720563]
  We evaluate 29 datasets using predictions from 18 pretrained Transformer models on individual test examples. We find that Quoref, HellaSwag, and MC-TACO are best suited for distinguishing among state-of-the-art models. We also observe span selection task format, which is used for QA datasets like QAMR or SQuAD2.0, is effective in differentiating between strong and weak models.
  arXiv  Detail & Related papers  (2021-06-01T22:33:53Z)
• An Experimental Review on Deep Learning Architectures for Time Series Forecasting [0.0]
  We provide the most extensive deep learning study for time series forecasting. Among all studied models, the results show that long short-term memory (LSTM) and convolutional networks (CNN) are the best alternatives. CNNs achieve comparable performance with less variability of results under different parameter configurations, while also being more efficient.
  arXiv  Detail & Related papers  (2021-03-22T17:58:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.