A SSM is Polymerized from Multivariate Time Series

• URL: http://arxiv.org/abs/2409.20310v2
• Date: Tue, 1 Oct 2024 03:32:24 GMT
• Title: A SSM is Polymerized from Multivariate Time Series
• Authors: Haixiang Wu,
• Abstract summary: We develop Poly-Mamba, a novel method for MTS forecasting. Experiments on six real-world datasets demonstrate that Poly-Mamba outperforms the SOTA methods.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: For multivariate time series (MTS) tasks, previous state space models (SSMs) followed the modeling paradigm of Transformer-based methods. However, none of them explicitly model the complex dependencies of MTS: the Channel Dependency variations with Time (CDT). In view of this, we delve into the derivation of SSM, which involves approximating continuously updated functions by orthogonal function basis. We then develop Poly-Mamba, a novel method for MTS forecasting. Its core concept is to expand the original orthogonal function basis space into a multivariate orthogonal function space containing variable mixing terms, and make a projection on this space so as to explicitly describe the CDT by weighted coefficients. In Poly-Mamba, we propose the Multivariate Orthogonal Polynomial Approximation (MOPA) as a simplified implementation of this concept. For the simple linear relationship between channels, we propose Linear Channel Mixing (LCM) and generate CDT patterns adaptively for different channels through a proposed Order Combining method. Experiments on six real-world datasets demonstrate that Poly-Mamba outperforms the SOTA methods, especially when dealing with datasets having a large number of channels and complex correlations. The codes and log files will be released at: https://github.com/Joeland4/Poly-Mamba.

Related papers

• DisenTS: Disentangled Channel Evolving Pattern Modeling for Multivariate Time Series Forecasting [43.071713191702486]
  DisenTS is a tailored framework for modeling disentangled channel evolving patterns in general time series forecasting. We introduce a novel Forecaster Aware Gate (FAG) module that generates the routing signals adaptively according to both the forecasters' states and input series' characteristics.
  arXiv  Detail & Related papers  (2024-10-30T12:46:14Z)
• VE: Modeling Multivariate Time Series Correlation with Variate Embedding [0.4893345190925178]
  Current channel-independent (CI) models and models with a CI final projection layer are unable to capture correlations. We present the variate embedding (VE) pipeline, which learns a unique and consistent embedding for each variate. The VE pipeline can be integrated into any model with a CI final projection layer to improve multivariate forecasting.
  arXiv  Detail & Related papers  (2024-09-10T02:49:30Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
  We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T14:39:28Z)
• Adaptive Multi-Scale Decomposition Framework for Time Series Forecasting [26.141054975797868]
  We propose a novel Adaptive Multi-Scale Decomposition (AMD) framework for time series forecasting (TSF) Our framework decomposes time series into distinct temporal patterns at multiple scales, leveraging the Multi-Scale Decomposable Mixing (MDM) block. Our approach effectively models both temporal and channel dependencies and utilizes autocorrelation to refine multi-scale data integration.
  arXiv  Detail & Related papers  (2024-06-06T05:27:33Z)
• SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion [59.96233305733875]
  Time series forecasting plays a crucial role in various fields such as finance, traffic management, energy, and healthcare. Several methods utilize mechanisms like attention or mixer to address this by capturing channel correlations. This paper presents an efficient-based model, the Series-cOre Fused Time Series forecaster (SOFTS)
  arXiv  Detail & Related papers  (2024-04-22T14:06:35Z)
• Sample Complexity Characterization for Linear Contextual MDPs [67.79455646673762]
  Contextual decision processes (CMDPs) describe a class of reinforcement learning problems in which the transition kernels and reward functions can change over time with different MDPs indexed by a context variable. CMDPs serve as an important framework to model many real-world applications with time-varying environments. We study CMDPs under two linear function approximation models: Model I with context-varying representations and common linear weights for all contexts; and Model II with common representations for all contexts and context-varying linear weights.
  arXiv  Detail & Related papers  (2024-02-05T03:25:04Z)
• Heterogeneous Multi-Task Gaussian Cox Processes [61.67344039414193]
  We present a novel extension of multi-task Gaussian Cox processes for modeling heterogeneous correlated tasks jointly. A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks. We derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters.
  arXiv  Detail & Related papers  (2023-08-29T15:01:01Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)
• MTHetGNN: A Heterogeneous Graph Embedding Framework for Multivariate Time Series Forecasting [4.8274015390665195]
  We propose a novel end-to-end deep learning model, termed Multivariate Time Series Forecasting via Heterogeneous Graph Neural Networks (MTHetGNN) To characterize complex relations among variables, a relation embedding module is designed in MTHetGNN, where each variable is regarded as a graph node. A temporal embedding module is introduced for time series features extraction, where involving convolutional neural network (CNN) filters with different perception scales.
  arXiv  Detail & Related papers  (2020-08-19T18:21:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.