xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

• URL: http://arxiv.org/abs/2410.16928v2
• Date: Wed, 23 Oct 2024 08:13:11 GMT
• Title: xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories
• Authors: Maurice Kraus, Felix Divo, Devendra Singh Dhami, Kristian Kersting,
• Abstract summary: Time series data is prevalent across numerous fields, necessitating the development of robust and accurate forecasting models. We introduce xLSTM-Mixer, a model designed to effectively integrate temporal sequences, joint time-variable information, and multiple perspectives for robust forecasting. Our evaluations demonstrate xLSTM-Mixer's superior long-term forecasting performance compared to recent state-of-the-art methods.
• Score: 20.773694998061707
• License:
• Abstract: Time series data is prevalent across numerous fields, necessitating the development of robust and accurate forecasting models. Capturing patterns both within and between temporal and multivariate components is crucial for reliable predictions. We introduce xLSTM-Mixer, a model designed to effectively integrate temporal sequences, joint time-variate information, and multiple perspectives for robust forecasting. Our approach begins with a linear forecast shared across variates, which is then refined by xLSTM blocks. These blocks serve as key elements for modeling the complex dynamics of challenging time series data. xLSTM-Mixer ultimately reconciles two distinct views to produce the final forecast. Our extensive evaluations demonstrate xLSTM-Mixer's superior long-term forecasting performance compared to recent state-of-the-art methods. A thorough model analysis provides further insights into its key components and confirms its robustness and effectiveness. This work contributes to the resurgence of recurrent models in time series forecasting.

Related papers

• LLM-Mixer: Multiscale Mixing in LLMs for Time Series Forecasting [0.08795040582681389]
  LLM-Mixer is a framework that improves forecasting accuracy through the combination of multiscale time-series decomposition with pre-trained LLMs. It captures both short-term fluctuations and long-term trends by decomposing the data into multiple temporal resolutions.
  arXiv  Detail & Related papers  (2024-10-15T15:08:57Z)
• MGCP: A Multi-Grained Correlation based Prediction Network for Multivariate Time Series [54.91026286579748]
  We propose a Multi-Grained Correlations-based Prediction Network. It simultaneously considers correlations at three levels to enhance prediction performance. It employs adversarial training with an attention mechanism-based predictor and conditional discriminator to optimize prediction results at coarse-grained level.
  arXiv  Detail & Related papers  (2024-05-30T03:32:44Z)
• TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting [19.88184356154215]
  Time series forecasting is widely used in applications, such as traffic planning and weather forecasting. TimeMixer is able to achieve consistent state-of-the-art performances in both long-term and short-term forecasting tasks.
  arXiv  Detail & Related papers  (2024-05-23T14:27:07Z)
• PDETime: Rethinking Long-Term Multivariate Time Series Forecasting from the perspective of partial differential equations [49.80959046861793]
  We present PDETime, a novel LMTF model inspired by the principles of Neural PDE solvers. Our experimentation across seven diversetemporal real-world LMTF datasets reveals that PDETime adapts effectively to the intrinsic nature of the data.
  arXiv  Detail & Related papers  (2024-02-25T17:39:44Z)
• RPMixer: Shaking Up Time Series Forecasting with Random Projections for Large Spatial-Temporal Data [33.0546525587517]
  We propose a all-Multi-Layer Perceptron (all-MLP) time series forecasting architecture called RPMixer. Our method capitalizes on the ensemble-like behavior of deep neural networks, where each individual block behaves like a base learner in an ensemble model.
  arXiv  Detail & Related papers  (2024-02-16T07:28:59Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)
• TSMixer: An All-MLP Architecture for Time Series Forecasting [41.178272171720316]
  Time-Series Mixer (TSMixer) is a novel architecture designed by stacking multi-layer perceptrons (MLPs) On popular academic benchmarks, the simple-to-implement TSMixer is comparable to specialized state-of-the-art models. We present various analyses to shed light into the capabilities of TSMixer.
  arXiv  Detail & Related papers  (2023-03-10T16:41:24Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
  We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF) Our model avoids the influence of cumulative error and does not increase the time complexity. Our model achieves state-of-the-art performance on many popular multivariate datasets.
  arXiv  Detail & Related papers  (2022-05-16T07:53:42Z)
• Synergetic Learning of Heterogeneous Temporal Sequences for Multi-Horizon Probabilistic Forecasting [48.8617204809538]
  We propose Variational Synergetic Multi-Horizon Network (VSMHN), a novel deep conditional generative model. To learn complex correlations across heterogeneous sequences, a tailored encoder is devised to combine the advances in deep point processes models and variational recurrent neural networks. Our model can be trained effectively using variational inference and generates predictions with Monte-Carlo simulation.
  arXiv  Detail & Related papers  (2021-01-31T11:00:55Z)
• Multivariate Probabilistic Time Series Forecasting via Conditioned Normalizing Flows [8.859284959951204]
  Time series forecasting is fundamental to scientific and engineering problems. Deep learning methods are well suited for this problem. We show that it improves over the state-of-the-art for standard metrics on many real-world data sets.
  arXiv  Detail & Related papers  (2020-02-14T16:16:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.