Logo-LLM: Local and Global Modeling with Large Language Models for Time Series Forecasting

• URL: http://arxiv.org/abs/2505.11017v1
• Date: Fri, 16 May 2025 09:10:49 GMT
• Title: Logo-LLM: Local and Global Modeling with Large Language Models for Time Series Forecasting
• Authors: Wenjie Ou, Zhishuo Zhao, Dongyue Guo, Yi Lin,
• Abstract summary: Time series forecasting is critical across multiple domains, where time series data exhibits both local patterns and global dependencies.<n>Recent methods that adapt large language models (LLMs) into time series forecasting inherit this limitation by treating LLMs as black-box encoders.<n>We propose Logo-LLM, a novel LLM-based framework that explicitly extracts and models multi-scale temporal features.
• Score: 6.537801724497769
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Time series forecasting is critical across multiple domains, where time series data exhibits both local patterns and global dependencies. While Transformer-based methods effectively capture global dependencies, they often overlook short-term local variations in time series. Recent methods that adapt large language models (LLMs) into time series forecasting inherit this limitation by treating LLMs as black-box encoders, relying solely on the final-layer output and underutilizing hierarchical representations. To address this limitation, we propose Logo-LLM, a novel LLM-based framework that explicitly extracts and models multi-scale temporal features from different layers of a pre-trained LLM. Through empirical analysis, we show that shallow layers of LLMs capture local dynamics in time series, while deeper layers encode global trends. Moreover, Logo-LLM introduces lightweight Local-Mixer and Global-Mixer modules to align and integrate features with the temporal input across layers. Extensive experiments demonstrate that Logo-LLM achieves superior performance across diverse benchmarks, with strong generalization in few-shot and zero-shot settings while maintaining low computational overhead.

Related papers

• PlaM: Training-Free Plateau-Guided Model Merging for Better Visual Grounding in MLLMs [59.78917775399492]
  Multimodal instruction fine-tuning paradoxically degrades this text's reasoning capability.<n>We propose a training-free framework to mitigate this degradation.
  arXiv  Detail & Related papers  (2026-01-12T15:27:51Z)
• A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models [85.30893355216486]
  We study how visual token redundancy evolves with different dMLLM architectures and tasks.<n>Our study reveals that visual redundancy emerges only in from-scratch dMLLMs while handling long-answer tasks.<n>Layer-skipping is promising for accelerating AR-to-diffusion dMLLMs, whereas progressive or late-step pruning is more effective for from-scratch dMLLMs.
  arXiv  Detail & Related papers  (2025-11-19T04:13:36Z)
• Integrating Time Series into LLMs via Multi-layer Steerable Embedding Fusion for Enhanced Forecasting [44.91360223102709]
  Time series (TS) data are ubiquitous across various application areas, rendering time series forecasting (TSF) a fundamental task.<n>Existing methods are inherently constrained by their shallow integration of TS information.<n>We propose the Multi-layer Steerable Embedding Fusion (MSEF) to mitigate the progressive loss of TS information in deeper layers.
  arXiv  Detail & Related papers  (2025-08-22T03:22:10Z)
• Semantic-Enhanced Time-Series Forecasting via Large Language Models [20.383296465541758]
  Time series forecasting plays a significant role in finance, energy, meteorology, and IoT applications.<n>Recent studies have leveraged the generalization capabilities of large language models (LLMs) to adapt to time series forecasting, achieving promising performance.<n>We propose a novel Semantic-Enhanced LLM (SE-LLM) that explores the inherent periodicity and anomalous characteristics of time series to embed into the semantic space.
  arXiv  Detail & Related papers  (2025-08-11T07:19:21Z)
• Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Semantic Alignment [32.41581846555808]
  Large Language Models (LLMs) have recently demonstrated impressive capabilities in natural language processing.<n>We propose TALON, a unified framework that enhances LLM-based forecasting by modeling temporal and enforcing semantic alignment.<n>Experiments on seven real-world benchmarks demonstrate that TALON achieves superior performance across all datasets.
  arXiv  Detail & Related papers  (2025-08-10T06:06:19Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring.<n>Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data.<n>We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Model [63.14883657299359]
  Multi-modal Large Language Models (MLLMs) integrate visual and linguistic reasoning to address complex tasks such as image captioning and visual question answering.<n> tuning MLLMs for downstream tasks encounters two key challenges: Task-Expert, where distribution shifts between pre-training and target datasets constrain target performance, and OpenWorld Stabilization, where catastrophic forgetting erases the model general knowledge.
  arXiv  Detail & Related papers  (2025-03-06T15:29:13Z)
• Large Language Models are Few-shot Multivariate Time Series Classifiers [23.045734479292356]
  Large Language Models (LLMs) have been extensively applied in time series analysis.<n>Yet, their utility in the few-shot classification (i.e., a crucial training scenario) is underexplored.<n>We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem.
  arXiv  Detail & Related papers  (2025-01-30T03:59:59Z)
• LLM-Mixer: Multiscale Mixing in LLMs for Time Series Forecasting [0.08795040582681389]
  LLM-Mixer is a framework that improves forecasting accuracy through the combination of multiscale time-series decomposition with pre-trained LLMs. It captures both short-term fluctuations and long-term trends by decomposing the data into multiple temporal resolutions.
  arXiv  Detail & Related papers  (2024-10-15T15:08:57Z)
• Towards Time Series Reasoning with LLMs [0.4369058206183195]
  We propose a novel multi-modal time-series LLM approach that learns generalizable information across various domains with powerful zero-shot performance.<n>We show that our model learns a latent representation that reflects specific time-series features, as well as outperforming GPT-4o on a set of zero-shot reasoning tasks.
  arXiv  Detail & Related papers  (2024-09-17T17:23:44Z)
• CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning [59.88924847995279]
  We propose a novel Cross-Modal LLM Fine-Tuning (CALF) framework for MTSF.<n>To reduce the distribution discrepancy, we develop the cross-modal match module.<n>CALF establishes state-of-the-art performance for both long-term and short-term forecasting tasks.
  arXiv  Detail & Related papers  (2024-03-12T04:04:38Z)
• Probing Multimodal Large Language Models for Global and Local Semantic Representations [57.25949445963422]
  We study which layers of Multimodal Large Language Models make the most effort to the global image information. In this study, we find that the intermediate layers of models can encode more global semantic information. We find that the topmost layers may excessively focus on local information, leading to a diminished ability to encode global information.
  arXiv  Detail & Related papers  (2024-02-27T08:27:15Z)
• Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities [46.02234423159257]
  Large language models (LLMs) have been applied in many fields and have developed rapidly in recent years.<n>Recent works treat large language models as emphzero-shot time series reasoners without further fine-tuning.<n>Our study shows that LLMs perform well in predicting time series with clear patterns and trends, but face challenges with datasets lacking periodicity.
  arXiv  Detail & Related papers  (2024-02-16T17:15:28Z)
• AutoTimes: Autoregressive Time Series Forecasters via Large Language Models [67.83502953961505]
  AutoTimes projects time series into the embedding space of language tokens and autoregressively generates future predictions with arbitrary lengths. We formulate time series as prompts, extending the context for prediction beyond the lookback window. AutoTimes achieves state-of-the-art with 0.1% trainable parameters and over $5times$ training/inference speedup.
  arXiv  Detail & Related papers  (2024-02-04T06:59:21Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.