Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer

• URL: http://arxiv.org/abs/2405.17478v3
• Date: Sun, 07 Sep 2025 06:08:47 GMT
• Title: Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
• Authors: Yihang Wang, Yuying Qiu, Peng Chen, Kai Zhao, Yang Shu, Zhongwen Rao, Lujia Pan, Bin Yang, Chenjuan Guo,
• Abstract summary: Existing time series foundation models primarily focus on scaling up pre-training datasets and model sizes to enhance generalization performance.<n>We take a different approach by addressing two critical aspects of general forecasting models: how to derive unified representations from heterogeneous multi-domain time series data, and how to effectively capture domain-specific features to enable adaptive transfer across various downstream scenarios.<n>Our model achieves the state-of-the-art forecasting performance on seven real-world benchmarks, demonstrating remarkable few-shot and zero-shot capabilities.
• Score: 24.03830611693476
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the growing availability of multi-domain time series data, there is an increasing demand for general forecasting models pre-trained on multi-source datasets to support diverse downstream prediction scenarios. Existing time series foundation models primarily focus on scaling up pre-training datasets and model sizes to enhance generalization performance. In this paper, we take a different approach by addressing two critical aspects of general forecasting models: (1) how to derive unified representations from heterogeneous multi-domain time series data, and (2) how to effectively capture domain-specific features to enable adaptive transfer across various downstream scenarios. To address the first aspect, we propose Decomposed Frequency Learning as the pre-training task, which leverages frequency-based masking and reconstruction to decompose coupled semantic information in time series, resulting in unified representations across domains. For the second aspect, we introduce the Time Series Register, which captures domain-specific representations during pre-training and enhances adaptive transferability to downstream tasks. Our model achieves the state-of-the-art forecasting performance on seven real-world benchmarks, demonstrating remarkable few-shot and zero-shot capabilities.

Related papers

• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• A Unified Frequency Domain Decomposition Framework for Interpretable and Robust Time Series Forecasting [81.73338008264115]
  Current approaches for time series forecasting, whether in the time or frequency domain, predominantly use deep learning models based on linear layers or transformers.<n>We propose FIRE, a unified frequency domain decomposition framework that provides a mathematical abstraction for diverse types of time series.<n>Fire consistently outperforms state-of-the-art models on long-term forecasting benchmarks.
  arXiv  Detail & Related papers  (2025-10-11T09:59:25Z)
• A Wireless Foundation Model for Multi-Task Prediction [50.21098141769079]
  We propose a unified foundation model for multi-task prediction in wireless networks that supports diverse prediction intervals.<n>After trained on large-scale datasets, the proposed foundation model demonstrates strong generalization to unseen scenarios and zero-shot performance on new tasks.
  arXiv  Detail & Related papers  (2025-07-08T12:37:55Z)
• Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach [7.631288333466648]
  We conduct the first large-scale study evaluating leading generative models in data-scarce settings.<n>We propose a unified diffusion-based generative framework that can synthesize high-fidelity time series using just a few examples.
  arXiv  Detail & Related papers  (2025-05-26T18:39:04Z)
• Bridging Distribution Gaps in Time Series Foundation Model Pretraining with Prototype-Guided Normalization [29.082583523943157]
  We propose a domain-aware adaptive normalization strategy within the Transformer architecture.<n>We replace the traditional LayerNorm with a prototype-guided dynamic normalization mechanism (ProtoNorm)<n>Our method significantly outperforms conventional pretraining techniques across both classification and forecasting tasks.
  arXiv  Detail & Related papers  (2025-04-15T06:23:00Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies. We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis. Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• Tackling Data Heterogeneity in Federated Time Series Forecasting [61.021413959988216]
  Time series forecasting plays a critical role in various real-world applications, including energy consumption prediction, disease transmission monitoring, and weather forecasting. Most existing methods rely on a centralized training paradigm, where large amounts of data are collected from distributed devices to a central cloud server. We propose a novel framework, Fed-TREND, to address data heterogeneity by generating informative synthetic data as auxiliary knowledge carriers.
  arXiv  Detail & Related papers  (2024-11-24T04:56:45Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a causal Transformer for unified time series forecasting. Based on large-scale pre-training, Timer-XL achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• Metadata Matters for Time Series: Informative Forecasting with Transformers [70.38241681764738]
  We propose a Metadata-informed Time Series Transformer (MetaTST) for time series forecasting. To tackle the unstructured nature of metadata, MetaTST formalizes them into natural languages by pre-designed templates. A Transformer encoder is employed to communicate series and metadata tokens, which can extend series representations by metadata information.
  arXiv  Detail & Related papers  (2024-10-04T11:37:55Z)
• UniCL: A Universal Contrastive Learning Framework for Large Time Series Models [18.005358506435847]
  Time-series analysis plays a pivotal role across a range of critical applications, from finance to healthcare. Traditional supervised learning methods first annotate extensive labels for time-series data in each task. This paper introduces UniCL, a universal and scalable contrastive learning framework designed for pretraining time-series foundation models.
  arXiv  Detail & Related papers  (2024-05-17T07:47:11Z)
• Generative Pretrained Hierarchical Transformer for Time Series Forecasting [3.739587363053192]
  We propose a novel generative pretrained hierarchical transformer architecture for forecasting, named textbfGPHT. We conduct sufficient experiments on eight datasets with mainstream self-supervised pretraining models and supervised models. The results demonstrated that GPHT surpasses the baseline models across various fine-tuning and zero/few-shot learning settings in the traditional long-term forecasting task.
  arXiv  Detail & Related papers  (2024-02-26T11:54:54Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Timer: Generative Pre-trained Transformers Are Large Time Series Models [83.03091523806668]
  This paper aims at the early development of large time series models (LTSM) During pre-training, we curate large-scale datasets with up to 1 billion time points. To meet diverse application needs, we convert forecasting, imputation, and anomaly detection of time series into a unified generative task.
  arXiv  Detail & Related papers  (2024-02-04T06:55:55Z)
• Large Pre-trained time series models for cross-domain Time series analysis tasks [20.228846068418765]
  We propose a novel method of textitadaptive segmentation that automatically identifies optimal dataset-specific segmentation strategy during pre-training. This enables LPTM to perform similar to or better than domain-specific state-of-art model when fine-tuned to different downstream time-series analysis tasks and under zero-shot settings.
  arXiv  Detail & Related papers  (2023-11-19T20:16:16Z)
• UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting [59.11817101030137]
  This research advocates for a unified model paradigm that transcends domain boundaries. Learning an effective cross-domain model presents the following challenges. We propose UniTime for effective cross-domain time series learning.
  arXiv  Detail & Related papers  (2023-10-15T06:30:22Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• Ti-MAE: Self-Supervised Masked Time Series Autoencoders [16.98069693152999]
  We propose a novel framework named Ti-MAE, in which the input time series are assumed to follow an integrate distribution. Ti-MAE randomly masks out embedded time series data and learns an autoencoder to reconstruct them at the point-level. Experiments on several public real-world datasets demonstrate that our framework of masked autoencoding could learn strong representations directly from the raw data.
  arXiv  Detail & Related papers  (2023-01-21T03:20:23Z)
• HyperTime: Implicit Neural Representation for Time Series [131.57172578210256]
  Implicit neural representations (INRs) have recently emerged as a powerful tool that provides an accurate and resolution-independent encoding of data. In this paper, we analyze the representation of time series using INRs, comparing different activation functions in terms of reconstruction accuracy and training convergence speed. We propose a hypernetwork architecture that leverages INRs to learn a compressed latent representation of an entire time series dataset.
  arXiv  Detail & Related papers  (2022-08-11T14:05:51Z)
• Time Series is a Special Sequence: Forecasting with Sample Convolution and Interaction [9.449017120452675]
  Time series is a special type of sequence data, a set of observations collected at even intervals of time and ordered chronologically. Existing deep learning techniques use generic sequence models for time series analysis, which ignore some of its unique properties. We propose a novel neural network architecture and apply it for the time series forecasting problem, wherein we conduct sample convolution and interaction at multiple resolutions for temporal modeling.
  arXiv  Detail & Related papers  (2021-06-17T08:15:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.