Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift

• URL: http://arxiv.org/abs/2410.09836v1
• Date: Sun, 13 Oct 2024 13:35:29 GMT
• Title: Learning Pattern-Specific Experts for Time Series Forecasting Under Patch-level Distribution Shift
• Authors: Yanru Sun, Zongxia Xie, Emadeldeen Eldele, Dongyue Chen, Qinghua Hu, Min Wu,
• Abstract summary: Time series forecasting aims to predict future values based on historical data. Real-world time often exhibit complex non-uniform distribution with varying patterns across segments, such as season, operating condition, or semantic meaning. We propose bftextS, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting.
• Score: 30.581736814767606
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Time series forecasting, which aims to predict future values based on historical data, has garnered significant attention due to its broad range of applications. However, real-world time series often exhibit complex non-uniform distribution with varying patterns across segments, such as season, operating condition, or semantic meaning, making accurate forecasting challenging. Existing approaches, which typically train a single model to capture all these diverse patterns, often struggle with the pattern drifts between patches and may lead to poor generalization. To address these challenges, we propose \textbf{TFPS}, a novel architecture that leverages pattern-specific experts for more accurate and adaptable time series forecasting. TFPS employs a dual-domain encoder to capture both time-domain and frequency-domain features, enabling a more comprehensive understanding of temporal dynamics. It then uses subspace clustering to dynamically identify distinct patterns across data patches. Finally, pattern-specific experts model these unique patterns, delivering tailored predictions for each patch. By explicitly learning and adapting to evolving patterns, TFPS achieves significantly improved forecasting accuracy. Extensive experiments on real-world datasets demonstrate that TFPS outperforms state-of-the-art methods, particularly in long-term forecasting, through its dynamic and pattern-aware learning approach. The data and codes are available: \url{https://github.com/syrGitHub/TFPS}.

Related papers

• FlexTSF: A Universal Forecasting Model for Time Series with Variable Regularities [17.164913785452367]
  We propose FlexTSF, a universal time series forecasting model that possesses better generalization and supports both regular and irregular time series. Experiments on 12 datasets show that FlexTSF outperforms state-of-the-art forecasting models respectively designed for regular and irregular time series.
  arXiv  Detail & Related papers  (2024-10-30T16:14:09Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a generative Transformer for unified time series forecasting. Timer-XL achieves state-of-the-art performance across challenging forecasting benchmarks through a unified approach.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• DAM: Towards A Foundation Model for Time Series Forecasting [0.8231118867997028]
  We propose a neural model that takes randomly sampled histories and outputs an adjustable basis composition as a continuous function of time. It involves three key components: (1) a flexible approach for using randomly sampled histories from a long-tail distribution; (2) a transformer backbone that is trained on these actively sampled histories to produce, as representational output; and (3) the basis coefficients of a continuous function of time.
  arXiv  Detail & Related papers  (2024-07-25T08:48:07Z)
• Generative Pretrained Hierarchical Transformer for Time Series Forecasting [3.739587363053192]
  We propose a novel generative pretrained hierarchical transformer architecture for forecasting, named textbfGPHT. We conduct sufficient experiments on eight datasets with mainstream self-supervised pretraining models and supervised models. The results demonstrated that GPHT surpasses the baseline models across various fine-tuning and zero/few-shot learning settings in the traditional long-term forecasting task.
  arXiv  Detail & Related papers  (2024-02-26T11:54:54Z)
• PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations [49.80959046861793]
  We present PDETime, a novel LMTF model inspired by the principles of Neural PDE solvers. Our experimentation across seven diversetemporal real-world LMTF datasets reveals that PDETime adapts effectively to the intrinsic nature of the data.
  arXiv  Detail & Related papers  (2024-02-25T17:39:44Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• Spatiotemporal-Linear: Towards Universal Multivariate Time Series Forecasting [10.404951989266191]
  We introduce the Spatio-Temporal- Linear (STL) framework. STL seamlessly integrates time-embedded and spatially-informed bypasses to augment the Linear-based architecture. Empirical evidence highlights STL's prowess, outpacing both Linear and Transformer benchmarks across varied observation and prediction durations and datasets.
  arXiv  Detail & Related papers  (2023-12-22T17:46:34Z)
• Deep Autoregressive Models with Spectral Attention [74.08846528440024]
  We propose a forecasting architecture that combines deep autoregressive models with a Spectral Attention (SA) module. By characterizing in the spectral domain the embedding of the time series as occurrences of a random process, our method can identify global trends and seasonality patterns. Two spectral attention models, global and local to the time series, integrate this information within the forecast and perform spectral filtering to remove time series's noise.
  arXiv  Detail & Related papers  (2021-07-13T11:08:47Z)
• Model-Attentive Ensemble Learning for Sequence Modeling [86.4785354333566]
  We present Model-Attentive Ensemble learning for Sequence modeling (MAES) MAES is a mixture of time-series experts which leverages an attention-based gating mechanism to specialize the experts on different sequence dynamics and adaptively weight their predictions. We demonstrate that MAES significantly out-performs popular sequence models on datasets subject to temporal shift.
  arXiv  Detail & Related papers  (2021-02-23T05:23:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.