Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning

• URL: http://arxiv.org/abs/2412.04806v1
• Date: Fri, 06 Dec 2024 06:32:47 GMT
• Title: Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning
• Authors: Jayanie Bogahawatte, Sachith Seneviratne, Maneesha Perera, Saman Halgamuge,
• Abstract summary: We propose NNCL-TLLM: Nearest Neighbor Contrastive Learning for Time series forecasting via Large Language Models.<n>First, we generate time series compatible text prototypes such that each text prototype represents both word token embeddings in its neighborhood and time series characteristics.<n>We then fine-tune the layer normalization and positional embeddings of the LLM, keeping the other layers intact, reducing the trainable parameters and decreasing the computational cost.
• Score: 1.7892194562398749
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Adapting Large Language Models (LLMs) that are extensively trained on abundant text data, and customizing the input prompt to enable time series forecasting has received considerable attention. While recent work has shown great potential for adapting the learned prior of LLMs, the formulation of the prompt to finetune LLMs remains challenging as prompt should be aligned with time series data. Additionally, current approaches do not effectively leverage word token embeddings which embody the rich representation space learned by LLMs. This emphasizes the need for a robust approach to formulate the prompt which utilizes the word token embeddings while effectively representing the characteristics of the time series. To address these challenges, we propose NNCL-TLLM: Nearest Neighbor Contrastive Learning for Time series forecasting via LLMs. First, we generate time series compatible text prototypes such that each text prototype represents both word token embeddings in its neighborhood and time series characteristics via end-to-end finetuning. Next, we draw inspiration from Nearest Neighbor Contrastive Learning to formulate the prompt while obtaining the top-$k$ nearest neighbor time series compatible text prototypes. We then fine-tune the layer normalization and positional embeddings of the LLM, keeping the other layers intact, reducing the trainable parameters and decreasing the computational cost. Our comprehensive experiments demonstrate that NNCL-TLLM outperforms in few-shot forecasting while achieving competitive or superior performance over the state-of-the-art methods in long-term and short-term forecasting tasks.

Related papers

• Enhancing Time Series Forecasting via Multi-Level Text Alignment with LLMs [6.612196783595362]
  We propose a multi-level text alignment framework for time series forecasting using large language models (LLMs) Our method decomposes time series into trend, seasonal, and residual components, which are then reprogrammed into component-specific text representations. Experiments on multiple datasets demonstrate that our method outperforms state-of-the-art models in accuracy while providing good interpretability.
  arXiv  Detail & Related papers  (2025-04-10T01:02:37Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring. Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data. We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop [63.34626300024294]
  TimeXL is a multi-modal prediction framework that integrates a prototype-based time series encoder. It produces more accurate predictions and interpretable explanations. Empirical evaluations on four real-world datasets demonstrate that TimeXL achieves up to 8.9% improvement in AUC.
  arXiv  Detail & Related papers  (2025-03-02T20:40:53Z)
• Real-time Verification and Refinement of Language Model Text Generation [60.04718679054704]
  Large language models (LLMs) have shown remarkable performance across a wide range of natural language tasks. A critical challenge remains in that they sometimes generate factually incorrect answers. We propose Streaming-VR, a novel approach designed to enhance the efficiency of verification and refinement of LLM outputs.
  arXiv  Detail & Related papers  (2025-01-14T03:59:48Z)
• TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment [21.690191536424567]
  TimeCMA is an intuitive yet effective framework for time series forecasting. Extensive experiments on eight real datasets demonstrate that TimeCMA outperforms state-of-the-arts.
  arXiv  Detail & Related papers  (2024-06-03T00:27:29Z)
• CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning [59.88924847995279]
  We propose a novel Cross-Modal LLM Fine-Tuning (CALF) framework for MTSF. To reduce the distribution discrepancy, we develop the cross-modal match module. CALF establishes state-of-the-art performance for both long-term and short-term forecasting tasks.
  arXiv  Detail & Related papers  (2024-03-12T04:04:38Z)
• $\textbf{S}^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting [21.921303835714628]
  We propose Semantic Space Informed Prompt learning with LLM ($S2$IP-LLM) to align the pre-trained semantic space with time series embeddings space. We show that the proposed $S2$IP-LLM can achieve superior forecasting performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-03-09T05:20:48Z)
• Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities [46.02234423159257]
  Large language models (LLMs) have been applied in many fields and have developed rapidly in recent years. Recent works treat large language models as emphzero-shot time series reasoners without further fine-tuning. Our study shows that LLMs perform well in predicting time series with clear patterns and trends, but face challenges with datasets lacking periodicity.
  arXiv  Detail & Related papers  (2024-02-16T17:15:28Z)
• AutoTimes: Autoregressive Time Series Forecasters via Large Language Models [67.83502953961505]
  AutoTimes projects time series into the embedding space of language tokens and autoregressively generates future predictions with arbitrary lengths. We formulate time series as prompts, extending the context for prediction beyond the lookback window. AutoTimes achieves state-of-the-art with 0.1% trainable parameters and over $5times$ training/inference speedup.
  arXiv  Detail & Related papers  (2024-02-04T06:59:21Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.