Evolving Multi-Scale Normalization for Time Series Forecasting under Distribution Shifts

• URL: http://arxiv.org/abs/2409.19718v1
• Date: Sun, 29 Sep 2024 14:26:22 GMT
• Title: Evolving Multi-Scale Normalization for Time Series Forecasting under Distribution Shifts
• Authors: Dalin Qin, Yehui Li, Weiqi Chen, Zhaoyang Zhu, Qingsong Wen, Liang Sun, Pierre Pinson, Yi Wang,
• Abstract summary: We propose a novel model-agnostic Evolving Multi-Scale Normalization (EvoMSN) framework to tackle the distribution shift problem. We evaluate the effectiveness of EvoMSN in improving the performance of five mainstream forecasting methods on benchmark datasets.
• Score: 20.02869280775877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Complex distribution shifts are the main obstacle to achieving accurate long-term time series forecasting. Several efforts have been conducted to capture the distribution characteristics and propose adaptive normalization techniques to alleviate the influence of distribution shifts. However, these methods neglect the intricate distribution dynamics observed from various scales and the evolving functions of distribution dynamics and normalized mapping relationships. To this end, we propose a novel model-agnostic Evolving Multi-Scale Normalization (EvoMSN) framework to tackle the distribution shift problem. Flexible normalization and denormalization are proposed based on the multi-scale statistics prediction module and adaptive ensembling. An evolving optimization strategy is designed to update the forecasting model and statistics prediction module collaboratively to track the shifting distributions. We evaluate the effectiveness of EvoMSN in improving the performance of five mainstream forecasting methods on benchmark datasets and also show its superiority compared to existing advanced normalization and online learning approaches. The code is publicly available at https://github.com/qindalin/EvoMSN.

Related papers

• MITA: Bridging the Gap between Model and Data for Test-time Adaptation [68.62509948690698]
  Test-Time Adaptation (TTA) has emerged as a promising paradigm for enhancing the generalizability of models. We propose Meet-In-The-Middle based MITA, which introduces energy-based optimization to encourage mutual adaptation of the model and data from opposing directions.
  arXiv  Detail & Related papers  (2024-10-12T07:02:33Z)
• Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift [40.734564394464556]
  We present a unified Probabilistic Graphical Model to Jointly capturing intra-/inter-series correlations and modeling the time-variant transitional distribution. We validate the effectiveness and efficiency of JointPGM through extensive experiments on six highly non-stationary MTS datasets.
  arXiv  Detail & Related papers  (2024-07-18T06:16:03Z)
• Low-rank Adaptation for Spatio-Temporal Forecasting [13.595533573828734]
  We present a novel low-rank adaptation framework as an off-the-shelf plugin for existing spatialtemporal prediction models, STLo-RA. Our approach increases parameters and training time of the original models by less than 4%, still achieving consistent and sustained performance enhancement.
  arXiv  Detail & Related papers  (2024-04-11T17:04:55Z)
• Addressing Distribution Shift in Time Series Forecasting with Instance Normalization Flows [36.956983415564274]
  We propose a general decoupled formulation for time series forecasting. We make such a formulation formalized into a bi-level optimization problem. Our method consistently outperforms state-of-the-art baselines on both synthetic and real-world data.
  arXiv  Detail & Related papers  (2024-01-30T06:35:52Z)
• Out-of-Distribution Generalized Dynamic Graph Neural Network with Disentangled Intervention and Invariance Promotion [61.751257172868186]
  Dynamic graph neural networks (DyGNNs) have demonstrated powerful predictive abilities by exploiting graph and temporal dynamics. Existing DyGNNs fail to handle distribution shifts, which naturally exist in dynamic graphs.
  arXiv  Detail & Related papers  (2023-11-24T02:42:42Z)
• Ensemble Modeling for Multimodal Visual Action Recognition [50.38638300332429]
  We propose an ensemble modeling approach for multimodal action recognition. We independently train individual modality models using a variant of focal loss tailored to handle the long-tailed distribution of the MECCANO [21] dataset.
  arXiv  Detail & Related papers  (2023-08-10T08:43:20Z)
• Distribution Shift Inversion for Out-of-Distribution Prediction [57.22301285120695]
  We propose a portable Distribution Shift Inversion algorithm for Out-of-Distribution (OoD) prediction. We show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms.
  arXiv  Detail & Related papers  (2023-06-14T08:00:49Z)
• Sinkhorn-Flow: Predicting Probability Mass Flow in Dynamical Systems Using Optimal Transport [89.61692654941106]
  We propose a new approach to predicting such mass flow over time using optimal transport. We apply our approach to the task of predicting how communities will evolve over time in social network settings.
  arXiv  Detail & Related papers  (2023-03-14T07:25:44Z)
• Scalable Dynamic Mixture Model with Full Covariance for Probabilistic Traffic Forecasting [16.04029885574568]
  We propose a dynamic mixture of zero-mean Gaussian distributions for the time-varying error process. The proposed method can be seamlessly integrated into existing deep-learning frameworks with only a few additional parameters to be learned. We evaluate the proposed method on a traffic speed forecasting task and find that our method not only improves model horizons but also provides interpretabletemporal correlation structures.
  arXiv  Detail & Related papers  (2022-12-10T22:50:00Z)
• Distributional Drift Adaptation with Temporal Conditional Variational Autoencoder for Multivariate Time Series Forecasting [41.206310481507565]
  We propose a novel framework temporal conditional variational autoencoder (TCVAE) to model the dynamic distributional dependencies over time. The TCVAE infers the dependencies as a temporal conditional distribution to leverage latent variables. We show the TCVAE's superior robustness and effectiveness over the state-of-the-art MTS forecasting baselines.
  arXiv  Detail & Related papers  (2022-09-01T10:06:22Z)
• An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation [91.62129090006745]
  This paper studies the distribution shift problem from the perspective of pre-training and data augmentation. We provide the first comprehensive empirical study focusing on pre-training and data augmentation.
  arXiv  Detail & Related papers  (2022-05-25T13:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.