Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- URL: http://arxiv.org/abs/2310.01728v2
- Date: Mon, 29 Jan 2024 06:27:53 GMT
- Title: Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
- Authors: Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming
Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, Qingsong Wen
- Abstract summary: Time series forecasting holds significant importance in many real-world dynamic systems.
We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting.
Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
- Score: 110.20279343734548
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Time series forecasting holds significant importance in many real-world
dynamic systems and has been extensively studied. Unlike natural language
process (NLP) and computer vision (CV), where a single large model can tackle
multiple tasks, models for time series forecasting are often specialized,
necessitating distinct designs for different tasks and applications. While
pre-trained foundation models have made impressive strides in NLP and CV, their
development in time series domains has been constrained by data sparsity.
Recent studies have revealed that large language models (LLMs) possess robust
pattern recognition and reasoning abilities over complex sequences of tokens.
However, the challenge remains in effectively aligning the modalities of time
series data and natural language to leverage these capabilities. In this work,
we present Time-LLM, a reprogramming framework to repurpose LLMs for general
time series forecasting with the backbone language models kept intact. We begin
by reprogramming the input time series with text prototypes before feeding it
into the frozen LLM to align the two modalities. To augment the LLM's ability
to reason with time series data, we propose Prompt-as-Prefix (PaP), which
enriches the input context and directs the transformation of reprogrammed input
patches. The transformed time series patches from the LLM are finally projected
to obtain the forecasts. Our comprehensive evaluations demonstrate that
Time-LLM is a powerful time series learner that outperforms state-of-the-art,
specialized forecasting models. Moreover, Time-LLM excels in both few-shot and
zero-shot learning scenarios.
Related papers
- Are Language Models Actually Useful for Time Series Forecasting? [21.378728572776897]
Large language models (LLMs) are being applied to time series tasks, particularly time series forecasting.
We find that removing the LLM component or replacing it with a basic attention layer does not degrade the forecasting results.
We also find that pretrained LLMs do no better than models trained from scratch, do not represent the sequential dependencies in time series, and do not assist in few-shot settings.
arXiv Detail & Related papers (2024-06-22T03:33:38Z) - TimeCMA: Towards LLM-Empowered Time Series Forecasting via Cross-Modality Alignment [21.690191536424567]
TimeCMA is a framework for time series forecasting with cross-modality alignment.
Extensive experiments on real data offer insight into the accuracy and efficiency of the proposed framework.
arXiv Detail & Related papers (2024-06-03T00:27:29Z) - Multi-Patch Prediction: Adapting LLMs for Time Series Representation
Learning [22.28251586213348]
aLLM4TS is an innovative framework that adapts Large Language Models (LLMs) for time-series representation learning.
A distinctive element of our framework is the patch-wise decoding layer, which departs from previous methods reliant on sequence-level decoding.
arXiv Detail & Related papers (2024-02-07T13:51:26Z) - Empowering Time Series Analysis with Large Language Models: A Survey [24.202539098675953]
We provide a systematic overview of methods that leverage large language models for time series analysis.
Specifically, we first state the challenges and motivations of applying language models in the context of time series.
Next, we categorize existing methods into different groups (i.e., direct query, tokenization, prompt design, fine-tune, and model integration) and highlight the key ideas within each group.
arXiv Detail & Related papers (2024-02-05T16:46:35Z) - Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
We present a Masked-based Universal Time Series Forecasting Transformer (Moirai)
Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains.
Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
arXiv Detail & Related papers (2024-02-04T20:00:45Z) - AutoTimes: Autoregressive Time Series Forecasters via Large Language Models [67.83502953961505]
We propose AutoTimes as autoregressive time series forecasters, which independently projects time series segments into the embedding space and autoregressively generates future predictions with arbitrary lengths.
AutoTimes achieves state-of-the-art with 0.1% trainable parameters and over 5 times training/inference speedup compared to advanced LLM-based forecasters.
arXiv Detail & Related papers (2024-02-04T06:59:21Z) - Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
This paper aims at the early development of large time series models (LTSM)
During pre-training, we curate large-scale datasets with up to 1 billion time points.
To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
arXiv Detail & Related papers (2024-02-04T06:55:55Z) - Lag-Llama: Towards Foundation Models for Probabilistic Time Series
Forecasting [54.04430089029033]
We present Lag-Llama, a general-purpose foundation model for time series forecasting based on a decoder-only transformer architecture.
Lag-Llama is pretrained on a large corpus of diverse time series data from several domains, and demonstrates strong zero-shot generalization capabilities.
When fine-tuned on relatively small fractions of such previously unseen datasets, Lag-Llama achieves state-of-the-art performance.
arXiv Detail & Related papers (2023-10-12T12:29:32Z) - Large Language Models Are Zero-Shot Time Series Forecasters [48.73953666153385]
By encoding time series as a string of numerical digits, we can frame time series forecasting as next-token prediction in text.
We find that large language models (LLMs) such as GPT-3 and LLaMA-2 can surprisingly zero-shot extrapolate time series at a level comparable to or exceeding the performance of purpose-built time series models trained on the downstream tasks.
arXiv Detail & Related papers (2023-10-11T19:01:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.