Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting

• URL: http://arxiv.org/abs/2405.14252v4
• Date: Tue, 08 Oct 2024 02:40:03 GMT
• Title: Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting
• Abstract summary: We propose Time-FFM, a Federated Foundation Model for Time series forecasting by leveraging pretrained LMs. Our comprehensive experiments indicate that Time-FFM outperforms state-of-the-arts and promises effective few-shot and zero-shot forecaster.
• Score: 43.56193749300678
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unlike natural language processing and computer vision, the development of Foundation Models (FMs) for time series forecasting is blocked due to data scarcity. While recent efforts are focused on building such FMs by unlocking the potential of language models (LMs) for time series analysis, dedicated parameters for various downstream forecasting tasks need training, which hinders the common knowledge sharing across domains. Moreover, data owners may hesitate to share the access to local data due to privacy concerns and copyright protection, which makes it impossible to simply construct a FM on cross-domain training instances. To address these issues, we propose Time-FFM, a Federated Foundation Model for Time series forecasting by leveraging pretrained LMs. Specifically, we begin by transforming time series into the modality of text tokens. To bootstrap LMs for time series reasoning, we propose a prompt adaption module to determine domain-customized prompts dynamically instead of artificially. Given the data heterogeneity across domains, we design a personalized federated training strategy by learning global encoders and local prediction heads. Our comprehensive experiments indicate that Time-FFM outperforms state-of-the-arts and promises effective few-shot and zero-shot forecaster.

Related papers

• LLM as Forecasting Planner: Training-Free Text Conditioning for Time-Series Foundation Models [7.741162056503623]
  Time-series foundation models (TSFMs) provide strong numerical forecasts, while large language models (LLMs) can reason over text.<n>We formulate forecasting as a planning problem over TSFM-generated trajectories.<n>We instantiate this as rc (textbfLLM textbfAs textbfForecasting textbfPlanner), a training-free framework that bridges the modality gap without retraining either model.
  arXiv  Detail & Related papers  (2026-07-27T15:02:40Z)
• Discrete Prototypical Memories for Federated Time Series Foundation Models [62.22804699052539]
  Leveraging Large Language Models as federated learning (FL)-based time series foundation models offers a promising way to transfer the generalization capabilities of LLMs to time series data.<n>However, the semantic misalignment between time-series data and the text-centric latent space of existing LLMs often leads to degraded performance.<n>We propose textscFeDPM, a federated framework for time-series foundation models based on discrete prototypical memories.
  arXiv  Detail & Related papers  (2026-04-06T06:57:08Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• Augmenting LLMs for General Time Series Understanding and Prediction [2.426309874608745]
  Time series data is fundamental to decision-making in many crucial domains including healthcare, finance, and environmental science.<n>We train this Time Series-augmented LLM (TsLLM) on a large corpus of over 2 million interleaved time series and text examples.<n>This training enables TsLLM to leverage both its language understanding and newly acquired temporal reasoning capabilities.
  arXiv  Detail & Related papers  (2025-10-01T16:54:46Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring. Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data. We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• TimesBERT: A BERT-Style Foundation Model for Time Series Understanding [72.64824086839631]
  GPT-style models have been positioned as foundation models for time series forecasting. BERT-style architecture has not been fully unlocked for time series understanding. We design TimesBERT to learn generic representations of time series. Our model is pre-trained on 260 billion time points across diverse domains.
  arXiv  Detail & Related papers  (2025-02-28T17:14:44Z)
• Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
  "Context is Key" (CiK) is a time series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings.
  arXiv  Detail & Related papers  (2024-10-24T17:56:08Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• A federated large language model for long-term time series forecasting [4.696083734269233]
  We propose FedTime, a federated large language model (LLM) tailored for long-range time series prediction. We employ K-means clustering to partition edge devices or clients into distinct clusters. We also incorporate channel independence and patching to better preserve local semantic information.
  arXiv  Detail & Related papers  (2024-07-30T02:38:27Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• Cross-Domain Pre-training with Language Models for Transferable Time Series Representations [32.8353465232791]
  CrossTimeNet is a novel cross-domain SSL learning framework to learn transferable knowledge from various domains. One of the key characteristics of CrossTimeNet is the newly designed time series tokenization module. We conduct extensive experiments in a real-world scenario across various time series classification domains.
  arXiv  Detail & Related papers  (2024-03-19T02:32:47Z)
• UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting [59.11817101030137]
  This research advocates for a unified model paradigm that transcends domain boundaries. Learning an effective cross-domain model presents the following challenges. We propose UniTime for effective cross-domain time series learning.
  arXiv  Detail & Related papers  (2023-10-15T06:30:22Z)
• Prompting-based Temporal Domain Generalization [10.377683220196873]
  This paper presents a novel prompting-based approach to temporal domain generalization. Our method adapts a trained model to temporal drift by learning global prompts, domain-specific prompts, and drift-aware prompts. Experiments on classification, regression, and time series forecasting tasks demonstrate the generality of the proposed approach.
  arXiv  Detail & Related papers  (2023-10-03T22:40:56Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• Self-Adaptive Forecasting for Improved Deep Learning on Non-Stationary Time-Series [20.958959332978726]
  SAF integrates a self-adaptation stage prior to forecasting based on backcasting' Our method enables efficient adaptation of encoded representations to evolving distributions, leading to superior generalization. On synthetic and real-world datasets in domains where time-series data are known to be notoriously non-stationary, such as healthcare and finance, we demonstrate a significant benefit of SAF.
  arXiv  Detail & Related papers  (2022-02-04T21:54:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.