Lightweight Channel-wise Dynamic Fusion Model: Non-stationary Time Series Forecasting via Entropy Analysis

• URL: http://arxiv.org/abs/2503.02609v1
• Date: Tue, 04 Mar 2025 13:29:42 GMT
• Title: Lightweight Channel-wise Dynamic Fusion Model: Non-stationary Time Series Forecasting via Entropy Analysis
• Authors: Tianyu Jia, Zongxia Xie, Yanru Sun, Dilfira Kudrat, Qinghua Hu,
• Abstract summary: We show that variance can be a valid and interpretable proxy for non-stationarity of time series.<n>We propose a novel lightweight textitChannel-wise textitDynamic textitFusion textitModel (textitCDFM)<n> Comprehensive experiments on seven time series datasets demonstrate the superiority and generalization capabilities of CDFM.
• Score: 25.291749176117662
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Non-stationarity is an intrinsic property of real-world time series and plays a crucial role in time series forecasting. Previous studies primarily adopt instance normalization to attenuate the non-stationarity of original series for better predictability. However, instance normalization that directly removes the inherent non-stationarity can lead to three issues: (1) disrupting global temporal dependencies, (2) ignoring channel-specific differences, and (3) producing over-smoothed predictions. To address these issues, we theoretically demonstrate that variance can be a valid and interpretable proxy for quantifying non-stationarity of time series. Based on the analysis, we propose a novel lightweight \textit{C}hannel-wise \textit{D}ynamic \textit{F}usion \textit{M}odel (\textit{CDFM}), which selectively and dynamically recovers intrinsic non-stationarity of the original series, while keeping the predictability of normalized series. First, we design a Dual-Predictor Module, which involves two branches: a Time Stationary Predictor for capturing stable patterns and a Time Non-stationary Predictor for modeling global dynamics patterns. Second, we propose a Fusion Weight Learner to dynamically characterize the intrinsic non-stationary information across different samples based on variance. Finally, we introduce a Channel Selector to selectively recover non-stationary information from specific channels by evaluating their non-stationarity, similarity, and distribution consistency, enabling the model to capture relevant dynamic features and avoid overfitting. Comprehensive experiments on seven time series datasets demonstrate the superiority and generalization capabilities of CDFM.

Related papers

• Predicting Forced Responses of Probability Distributions via the Fluctuation-Dissipation Theorem and Generative Modeling [0.0]
  We present a novel data-driven framework for estimating the response of higher-order moments of nonlinear systems to small external perturbations. Standard implementations rely on Gaussian approximations, which can often accurately predict the mean response but usually introduce significant biases in higher-order moments. We combine GFDT with recent advances in score-basedgenerative modeling, which enable direct estimation of the score function without requiring full density reconstruction.
  arXiv  Detail & Related papers  (2025-04-17T20:54:33Z)
• MFRS: A Multi-Frequency Reference Series Approach to Scalable and Accurate Time-Series Forecasting [51.94256702463408]
  Time series predictability is derived from periodic characteristics at different frequencies. We propose a novel time series forecasting method based on multi-frequency reference series correlation analysis. Experiments on major open and synthetic datasets show state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-03-11T11:40:14Z)
• Timer-XL: Long-Context Transformers for Unified Time Series Forecasting [67.83502953961505]
  We present Timer-XL, a causal Transformer for unified time series forecasting.<n>Based on large-scale pre-training, Timer-XL achieves state-of-the-art zero-shot performance.
  arXiv  Detail & Related papers  (2024-10-07T07:27:39Z)
• TwinS: Revisiting Non-Stationarity in Multivariate Time Series Forecasting [29.208648615140575]
  We propose a Transformer-based TwinS model to address the non-stationary periodic distributions. The Wavelet Convolution models nested periods by scaling the convolution kernel size like wavelet. The Period-Aware Attention guides attention by generating period relevance scores through a convolutional sub-network. The Channel-Temporal Mixed captures the overall relationships between time series through channel-time mixing learning.
  arXiv  Detail & Related papers  (2024-06-06T03:14:23Z)
• A Poisson-Gamma Dynamic Factor Model with Time-Varying Transition Dynamics [51.147876395589925]
  A non-stationary PGDS is proposed to allow the underlying transition matrices to evolve over time. A fully-conjugate and efficient Gibbs sampler is developed to perform posterior simulation. Experiments show that, in comparison with related models, the proposed non-stationary PGDS achieves improved predictive performance.
  arXiv  Detail & Related papers  (2024-02-26T04:39:01Z)
• Nonstationary Time Series Forecasting via Unknown Distribution Adaptation [22.915008205203886]
  As environments evolve, temporal distribution shifts can degrade time series forecasting performance. Some methods disentangle stationary and nonstationary components by assuming uniform distribution shifts, but it is impractical since when the distribution changes is unknown. We propose the textbfUnknown textbfDistribution textbfAdaptation (textbfUDA) model for nonstationary time series forecasting.
  arXiv  Detail & Related papers  (2024-02-20T07:16:12Z)
• IN-Flow: Instance Normalization Flow for Non-stationary Time Series Forecasting [38.4809915448213]
  We propose a decoupled formulation for time series forecasting with no reliance on fixed statistics.<n>We also propose instance normalization flow (IN-Flow), a novel invertible network for time series transformation.
  arXiv  Detail & Related papers  (2024-01-30T06:35:52Z)
• Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement [51.55157852647306]
  Time series forecasting has been a widely explored task of great importance in many applications. It is common that real-world time series data are recorded in a short time period, which results in a big gap between the deep model and the limited and noisy time series. We propose to address the time series forecasting problem with generative modeling and propose a bidirectional variational auto-encoder equipped with diffusion, denoise, and disentanglement.
  arXiv  Detail & Related papers  (2023-01-08T12:20:46Z)
• DRAformer: Differentially Reconstructed Attention Transformer for Time-Series Forecasting [7.805077630467324]
  Time-series forecasting plays an important role in many real-world scenarios, such as equipment life cycle forecasting, weather forecasting, and traffic flow forecasting. It can be observed from recent research that a variety of transformer-based models have shown remarkable results in time-series forecasting. However, there are still some issues that limit the ability of transformer-based models on time-series forecasting tasks.
  arXiv  Detail & Related papers  (2022-06-11T10:34:29Z)
• Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting [86.33543833145457]
  We propose Non-stationary Transformers as a generic framework with two interdependent modules: Series Stationarization and De-stationary Attention. Our framework consistently boosts mainstream Transformers by a large margin, which reduces MSE by 49.43% on Transformer, 47.34% on Informer, and 46.89% on Reformer.
  arXiv  Detail & Related papers  (2022-05-28T12:27:27Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Stochastically forced ensemble dynamic mode decomposition for forecasting and analysis of near-periodic systems [65.44033635330604]
  We introduce a novel load forecasting method in which observed dynamics are modeled as a forced linear system. We show that its use of intrinsic linear dynamics offers a number of desirable properties in terms of interpretability and parsimony. Results are presented for a test case using load data from an electrical grid.
  arXiv  Detail & Related papers  (2020-10-08T20:25:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.