xLSTMTime : Long-term Time Series Forecasting With xLSTM

• URL: http://arxiv.org/abs/2407.10240v3
• Date: Mon, 12 Aug 2024 02:10:34 GMT
• Title: xLSTMTime : Long-term Time Series Forecasting With xLSTM
• Authors: Musleh Alharthi, Ausif Mahmood,
• Abstract summary: This paper presents an adaptation of a recent architecture termed extended LSTM (xLSTM) for time series forecasting. We compare xLSTMTime's performance against various state-of-the-art models across multiple real-world da-tasets. Our findings suggest that refined recurrent architectures can offer competitive alternatives to transformer-based models in time series forecasting.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, transformer-based models have gained prominence in multivariate long-term time series forecasting (LTSF), demonstrating significant advancements despite facing challenges such as high computational demands, difficulty in capturing temporal dynamics, and managing long-term dependencies. The emergence of LTSF-Linear, with its straightforward linear architecture, has notably outperformed transformer-based counterparts, prompting a reevaluation of the transformer's utility in time series forecasting. In response, this paper presents an adaptation of a recent architecture termed extended LSTM (xLSTM) for LTSF. xLSTM incorporates exponential gating and a revised memory structure with higher capacity that has good potential for LTSF. Our adopted architecture for LTSF termed as xLSTMTime surpasses current approaches. We compare xLSTMTime's performance against various state-of-the-art models across multiple real-world da-tasets, demonstrating superior forecasting capabilities. Our findings suggest that refined recurrent architectures can offer competitive alternatives to transformer-based models in LTSF tasks, po-tentially redefining the landscape of time series forecasting.

Related papers

• QuLTSF: Long-Term Time Series Forecasting with Quantum Machine Learning [4.2117721107606005]
  Long-term time series forecasting involves predicting a large number of future values of a time series based on the past values. Recent quantum machine learning (QML) is evolving as a domain to enhance the capabilities of classical machine learning models. We show the advantages of QuLTSF over the state-of-the-art classical linear models, in terms of reduced mean squared error and mean absolute error.
  arXiv  Detail & Related papers  (2024-12-18T12:06:52Z)
• LSEAttention is All You Need for Time Series Forecasting [0.0]
  Transformer-based architectures have achieved remarkable success in natural language processing and computer vision. Previous research has identified the traditional attention mechanism as a key factor limiting their effectiveness in this domain. We introduce LATST, a novel approach designed to mitigate entropy collapse and training instability common challenges in Transformer-based time series forecasting.
  arXiv  Detail & Related papers  (2024-10-31T09:09:39Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a generative Transformer for unified time series forecasting. Timer-XL achieves state-of-the-art performance across challenging forecasting benchmarks through a unified approach.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• Unlocking the Power of LSTM for Long Term Time Series Forecasting [27.245021350821638]
  We propose a simple yet efficient algorithm named P-sLSTM built upon sLSTM by incorporating patching and channel independence. These modifications substantially enhance sLSTM's performance in TSF, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2024-08-19T13:59:26Z)
• Understanding Different Design Choices in Training Large Time Series Models [71.20102277299445]
  Training Large Time Series Models (LTSMs) on heterogeneous time series data poses unique challenges. We propose emphtime series prompt, a novel statistical prompting strategy tailored to time series data. We introduce textttLTSM-bundle, which bundles the best design choices we have identified.
  arXiv  Detail & Related papers  (2024-06-20T07:09:19Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• Transformers versus LSTMs for electronic trading [0.0]
  This study investigates whether Transformer-based model can be applied in financial time series prediction and beat LSTM. A new LSTM-based model called DLSTM is built and new architecture for the Transformer-based model is designed to adapt for financial prediction. The experiment result reflects that the Transformer-based model only has the limited advantage in absolute price sequence prediction.
  arXiv  Detail & Related papers  (2023-09-20T15:25:43Z)
• FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification [53.55504611255664]
  FormerTime is a hierarchical representation model for improving the classification capacity for the multivariate time series classification task. It exhibits three aspects of merits: (1) learning hierarchical multi-scale representations from time series data, (2) inheriting the strength of both transformers and convolutional networks, and (3) tacking the efficiency challenges incurred by the self-attention mechanism.
  arXiv  Detail & Related papers  (2023-02-20T07:46:14Z)
• Towards Long-Term Time-Series Forecasting: Feature, Pattern, and Distribution [57.71199089609161]
  Long-term time-series forecasting (LTTF) has become a pressing demand in many applications, such as wind power supply planning. Transformer models have been adopted to deliver high prediction capacity because of the high computational self-attention mechanism. We propose an efficient Transformerbased model, named Conformer, which differentiates itself from existing methods for LTTF in three aspects.
  arXiv  Detail & Related papers  (2023-01-05T13:59:29Z)
• CLMFormer: Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System [46.39662315849883]
  Long-term time-series forecasting (LTSF) plays a crucial role in various practical applications. Existing Transformer-based models, such as Fedformer and Informer, often achieve their best performances on validation sets after just a few epochs. We propose a novel approach to address this issue by employing curriculum learning and introducing a memory-driven decoder.
  arXiv  Detail & Related papers  (2022-07-16T04:05:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.