A decoder-only foundation model for time-series forecasting

• URL: http://arxiv.org/abs/2310.10688v4
• Date: Wed, 17 Apr 2024 18:24:45 GMT
• Title: A decoder-only foundation model for time-series forecasting
• Authors: Abhimanyu Das, Weihao Kong, Rajat Sen, Yichen Zhou,
• Abstract summary: Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus. It can work well across different forecasting history lengths, prediction lengths and temporal granularities.
• Score: 23.824504640087753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public datasets comes close to the accuracy of state-of-the-art supervised forecasting models for each individual dataset. Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus, and can work well across different forecasting history lengths, prediction lengths and temporal granularities.

Related papers

• In-Context Fine-Tuning for Time-Series Foundation Models [18.348874079298298]
  In particular, we design a pretrained foundation model that can be prompted with multiple time-series examples. Our foundation model is specifically trained to utilize examples from multiple related time-series in its context window. We show that such a foundation model that uses in-context examples at inference time can obtain much better performance on popular forecasting benchmarks.
  arXiv  Detail & Related papers  (2024-10-31T16:20:04Z)
• GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation [90.53485251837235]
  GIFT-Eval is a pioneering benchmark aimed at promoting evaluation across diverse datasets. GIFT-Eval encompasses 28 datasets over 144,000 time series and 177 million data points. We also provide a non-leaking pretraining dataset containing approximately 230 billion data points.
  arXiv  Detail & Related papers  (2024-10-14T11:29:38Z)
• DAM: Towards A Foundation Model for Time Series Forecasting [0.8231118867997028]
  We propose a neural model that takes randomly sampled histories and outputs an adjustable basis composition as a continuous function of time. It involves three key components: (1) a flexible approach for using randomly sampled histories from a long-tail distribution; (2) a transformer backbone that is trained on these actively sampled histories to produce, as representational output; and (3) the basis coefficients of a continuous function of time.
  arXiv  Detail & Related papers  (2024-07-25T08:48:07Z)
• Scaling Law for Time Series Forecasting [8.967263259533036]
  Scaling law that rewards large datasets, complex models and enhanced data granularity has been observed in various fields of deep learning. Yet, studies on time series forecasting have cast doubt on scaling behaviors of deep learning methods for time series forecasting. We propose a theory for scaling law for time series forecasting that can explain these seemingly abnormal behaviors.
  arXiv  Detail & Related papers  (2024-05-24T00:46:27Z)
• Chronos: Learning the Language of Time Series [79.38691251254173]
  Chronos is a framework for pretrained probabilistic time series models. We show that Chronos models can leverage time series data from diverse domains to improve zero-shot accuracy on unseen forecasting tasks.
  arXiv  Detail & Related papers  (2024-03-12T16:53:54Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecasting [54.04430089029033]
  We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture. Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities. When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-10-12T12:29:32Z)
• Pushing the Limits of Pre-training for Time Series Forecasting in the CloudOps Domain [54.67888148566323]
  We introduce three large-scale time series forecasting datasets from the cloud operations domain. We show it is a strong zero-shot baseline and benefits from further scaling, both in model and dataset size. Accompanying these datasets and results is a suite of comprehensive benchmark results comparing classical and deep learning baselines to our pre-trained method.
  arXiv  Detail & Related papers  (2023-10-08T08:09:51Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting [11.670324826998968]
  In existing time series forecasting methods, the models take a sequence of numerical values as input and yield numerical values as output. Inspired by the successes of pre-trained language foundation models, we propose a new forecasting paradigm: prompt-based time series forecasting. In this novel task, the numerical input and output are transformed into prompts and the forecasting task is framed in a sentence-to-sentence manner.
  arXiv  Detail & Related papers  (2022-09-20T10:15:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.