C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting
- URL: http://arxiv.org/abs/2406.05316v1
- Date: Sat, 8 Jun 2024 01:32:44 GMT
- Title: C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting
- Authors: Chaolv Zeng, Zhanyu Liu, Guanjie Zheng, Linghe Kong,
- Abstract summary: We propose textbfC-Mamba, a novel approach that captures cross-channel dependencies while maintaining linear complexity without losing the global receptive field.
Our model achieves state-of-the-art performance on seven real-world time series datasets.
- Score: 18.50360049235537
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, significant progress has been made in multivariate time series forecasting using Linear-based, Transformer-based, and Convolution-based models. However, these approaches face notable limitations: linear forecasters struggle with representation capacities, attention mechanisms suffer from quadratic complexity, and convolutional models have a restricted receptive field. These constraints impede their effectiveness in modeling complex time series, particularly those with numerous variables. Additionally, many models adopt the Channel-Independent (CI) strategy, treating multivariate time series as uncorrelated univariate series while ignoring their correlations. For models considering inter-channel relationships, whether through the self-attention mechanism, linear combination, or convolution, they all incur high computational costs and focus solely on weighted summation relationships, neglecting potential proportional relationships between channels. In this work, we address these issues by leveraging the newly introduced state space model and propose \textbf{C-Mamba}, a novel approach that captures cross-channel dependencies while maintaining linear complexity without losing the global receptive field. Our model consists of two key components: (i) channel mixup, where two channels are mixed to enhance the training sets; (ii) channel attention enhanced patch-wise Mamba encoder that leverages the ability of the state space models to capture cross-time dependencies and models correlations between channels by mining their weight relationships. Our model achieves state-of-the-art performance on seven real-world time series datasets. Moreover, the proposed mixup and attention strategy exhibits strong generalizability across other frameworks.
Related papers
- Channel-Aware Low-Rank Adaptation in Time Series Forecasting [43.684035409535696]
Two representative channel strategies are closely associated with model expressivity and robustness.
We present a channel-aware low-rank adaptation method to condition CD models on identity-aware individual components.
arXiv Detail & Related papers (2024-07-24T13:05:17Z) - UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens.
Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
arXiv Detail & Related papers (2024-06-07T14:39:28Z) - Chimera: Effectively Modeling Multivariate Time Series with 2-Dimensional State Space Models [5.37935922811333]
State Space Models (SSMs) are classical approaches for univariate time series modeling.
We present Chimera that uses two input-dependent 2-D SSM heads with different discretization processes to learn long-term progression and seasonal patterns.
Our experimental evaluation shows the superior performance of Chimera on extensive and diverse benchmarks.
arXiv Detail & Related papers (2024-06-06T17:58:09Z) - SOFTS: Efficient Multivariate Time Series Forecasting with Series-Core Fusion [59.96233305733875]
Time series forecasting plays a crucial role in various fields such as finance, traffic management, energy, and healthcare.
Several methods utilize mechanisms like attention or mixer to address this by capturing channel correlations.
This paper presents an efficient-based model, the Series-cOre Fused Time Series forecaster (SOFTS)
arXiv Detail & Related papers (2024-04-22T14:06:35Z) - From Similarity to Superiority: Channel Clustering for Time Series Forecasting [61.96777031937871]
We develop a novel and adaptable Channel Clustering Module ( CCM)
CCM dynamically groups channels characterized by intrinsic similarities and leverages cluster identity instead of channel identity.
CCM can boost the performance of CI and CD models by an average margin of 2.4% and 7.2% on long-term and short-term forecasting, respectively.
arXiv Detail & Related papers (2024-03-31T02:46:27Z) - MCformer: Multivariate Time Series Forecasting with Mixed-Channels Transformer [8.329947472853029]
Channel Independence (CI) strategy treats all channels as a single channel, expanding the dataset.
Mixed Channels strategy combines the data expansion advantages of the CI strategy with the ability to counteract inter-channel correlation forgetting.
Model blends a specific number of channels, leveraging an attention mechanism to effectively capture inter-channel correlation information.
arXiv Detail & Related papers (2024-03-14T09:43:07Z) - Enhancing Multivariate Time Series Forecasting with Mutual
Information-driven Cross-Variable and Temporal Modeling [24.041263835195423]
We introduce the Cross-variable Decorrelation Aware feature Modeling (CDAM) for Channel-mixing approaches.
We also introduce the Temporal correlation Aware Modeling (TAM) to exploit temporal correlations, a step beyond conventional single-step forecasting methods.
Our novel framework significantly surpasses existing models, including those previously considered state-of-the-art, in comprehensive tests.
arXiv Detail & Related papers (2024-03-01T04:42:47Z) - The Capacity and Robustness Trade-off: Revisiting the Channel
Independent Strategy for Multivariate Time Series Forecasting [50.48888534815361]
We show that models trained with the Channel Independent (CI) strategy outperform those trained with the Channel Dependent (CD) strategy.
Our results conclude that the CD approach has higher capacity but often lacks robustness to accurately predict distributionally drifted time series.
We propose a modified CD method called Predict Residuals with Regularization (PRReg) that can surpass the CI strategy.
arXiv Detail & Related papers (2023-04-11T13:15:33Z) - Multi-scale Attention Flow for Probabilistic Time Series Forecasting [68.20798558048678]
We propose a novel non-autoregressive deep learning model, called Multi-scale Attention Normalizing Flow(MANF)
Our model avoids the influence of cumulative error and does not increase the time complexity.
Our model achieves state-of-the-art performance on many popular multivariate datasets.
arXiv Detail & Related papers (2022-05-16T07:53:42Z) - Convolutional Tensor-Train LSTM for Spatio-temporal Learning [116.24172387469994]
We propose a higher-order LSTM model that can efficiently learn long-term correlations in the video sequence.
This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time.
Our results achieve state-of-the-art performance-art in a wide range of applications and datasets.
arXiv Detail & Related papers (2020-02-21T05:00:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.