When and How: Learning Identifiable Latent States for Nonstationary Time Series Forecasting

• URL: http://arxiv.org/abs/2402.12767v3
• Date: Fri, 7 Jun 2024 11:11:31 GMT
• Title: When and How: Learning Identifiable Latent States for Nonstationary Time Series Forecasting
• Authors: Zijian Li, Ruichu Cai, Zhenhui Yang, Haiqin Huang, Guangyi Chen, Yifan Shen, Zhengming Chen, Xiangchen Song, Kun Zhang,
• Abstract summary: We learn IDentifiable latEnt stAtes (IDEA) to detect when the distribution shifts occur. We further disentangle the stationary and nonstationary latent states via sufficient observation assumption to learn how the latent states change. Based on these theories, we devise the IDEA model, which incorporates an autoregressive hidden Markov model to estimate latent environments.
• Score: 22.915008205203886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Temporal distribution shifts are ubiquitous in time series data. One of the most popular methods assumes that the temporal distribution shift occurs uniformly to disentangle the stationary and nonstationary dependencies. But this assumption is difficult to meet, as we do not know when the distribution shifts occur. To solve this problem, we propose to learn IDentifiable latEnt stAtes (IDEA) to detect when the distribution shifts occur. Beyond that, we further disentangle the stationary and nonstationary latent states via sufficient observation assumption to learn how the latent states change. Specifically, we formalize the causal process with environment-irrelated stationary and environment-related nonstationary variables. Under mild conditions, we show that latent environments and stationary/nonstationary variables are identifiable. Based on these theories, we devise the IDEA model, which incorporates an autoregressive hidden Markov model to estimate latent environments and modular prior networks to identify latent states. The IDEA model outperforms several latest nonstationary forecasting methods on various benchmark datasets, highlighting its advantages in real-world scenarios.

Related papers

• Lightweight Channel-wise Dynamic Fusion Model: Non-stationary Time Series Forecasting via Entropy Analysis [25.291749176117662]
  We show that variance can be a valid and interpretable proxy for non-stationarity of time series. We propose a novel lightweight textitChannel-wise textitDynamic textitFusion textitModel (textitCDFM) Comprehensive experiments on seven time series datasets demonstrate the superiority and generalization capabilities of CDFM.
  arXiv  Detail & Related papers  (2025-03-04T13:29:42Z)
• Causal Temporal Representation Learning with Nonstationary Sparse Transition [22.6420431022419]
  Causal Temporal Representation Learning (Ctrl) methods aim to identify the temporal causal dynamics of complex nonstationary temporal sequences. This work adopts a sparse transition assumption, aligned with intuitive human understanding, and presents identifiability results from a theoretical perspective. We introduce a novel framework, Causal Temporal Representation Learning with Nonstationary Sparse Transition (CtrlNS), designed to leverage the constraints on transition sparsity.
  arXiv  Detail & Related papers  (2024-09-05T00:38:27Z)
• On the Identification of Temporally Causal Representation with Instantaneous Dependence [50.14432597910128]
  Temporally causal representation learning aims to identify the latent causal process from time series observations. Most methods require the assumption that the latent causal processes do not have instantaneous relations. We propose an textbfIDentification framework for instantanetextbfOus textbfLatent dynamics.
  arXiv  Detail & Related papers  (2024-05-24T08:08:05Z)
• Invariant Subspace Decomposition [10.655331762491613]
  We propose a novel framework for linear conditionals that splits the conditional distribution into a time-invariant and a residual time-dependent component. We show that this decomposition can be utilized both for zero-shot and time-adaptation prediction tasks. We propose a practical estimation procedure, which automatically infers the decomposition using tools from approximate joint matrix diagonalization.
  arXiv  Detail & Related papers  (2024-04-15T17:39:44Z)
• Proxy Methods for Domain Adaptation [78.03254010884783]
  proxy variables allow for adaptation to distribution shift without explicitly recovering or modeling latent variables. We develop a two-stage kernel estimation approach to adapt to complex distribution shifts in both settings.
  arXiv  Detail & Related papers  (2024-03-12T09:32:41Z)
• Probabilistic Learning of Multivariate Time Series with Temporal Irregularity [25.91078012394032]
  temporal irregularities, including nonuniform time intervals and component misalignment. We develop a conditional flow representation to non-parametrically represent the data distribution, which is typically non-Gaussian. The broad applicability and superiority of the proposed solution are confirmed by comparing it with existing approaches through ablation studies and testing on real-world datasets.
  arXiv  Detail & Related papers  (2023-06-15T14:08:48Z)
• Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shift [50.98086766507025]
  We propose a simple-yet-effective data augmentation strategy, Adversarial Invariant Augmentation (AIA) AIA aims to extrapolate and generate new environments, while concurrently preserving the original stable features during the augmentation process.
  arXiv  Detail & Related papers  (2022-11-05T07:55:55Z)
• Temporally Disentangled Representation Learning [14.762231867144065]
  It is unknown whether the underlying latent variables and their causal relations are identifiable if they have arbitrary, nonparametric causal influences in between. We propose textbftextttTDRL, a principled framework to recover time-delayed latent causal variables. Our approach considerably outperforms existing baselines that do not correctly exploit this modular representation of changes.
  arXiv  Detail & Related papers  (2022-10-24T23:02:49Z)
• DynaConF: Dynamic Forecasting of Non-Stationary Time Series [4.286546152336783]
  We propose a new method to model non-stationary conditional distributions over time. We show that our model can adapt to non-stationary time series better than state-of-the-art deep learning solutions.
  arXiv  Detail & Related papers  (2022-09-17T21:40:02Z)
• Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations [84.42837346400151]
  Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare. Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions. We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
  arXiv  Detail & Related papers  (2022-06-16T17:15:15Z)
• Weakly Supervised Representation Learning with Sparse Perturbations [82.39171485023276]
  We show that if one has weak supervision from observations generated by sparse perturbations of the latent variables, identification is achievable under unknown continuous latent distributions. We propose a natural estimation procedure based on this theory and illustrate it on low-dimensional synthetic and image-based experiments.
  arXiv  Detail & Related papers  (2022-06-02T15:30:07Z)
• Differentiable Invariant Causal Discovery [106.87950048845308]
  Learning causal structure from observational data is a fundamental challenge in machine learning. This paper proposes Differentiable Invariant Causal Discovery (DICD) to avoid learning spurious edges and wrong causal directions. Extensive experiments on synthetic and real-world datasets verify that DICD outperforms state-of-the-art causal discovery methods up to 36% in SHD.
  arXiv  Detail & Related papers  (2022-05-31T09:29:07Z)
• Certifying Model Accuracy under Distribution Shifts [151.67113334248464]
  We present provable robustness guarantees on the accuracy of a model under bounded Wasserstein shifts of the data distribution. We show that a simple procedure that randomizes the input of the model within a transformation space is provably robust to distributional shifts under the transformation.
  arXiv  Detail & Related papers  (2022-01-28T22:03:50Z)
• Tracking the risk of a deployed model and detecting harmful distribution shifts [105.27463615756733]
  In practice, it may make sense to ignore benign shifts, under which the performance of a deployed model does not degrade substantially. We argue that a sensible method for firing off a warning has to both (a) detect harmful shifts while ignoring benign ones, and (b) allow continuous monitoring of model performance without increasing the false alarm rate.
  arXiv  Detail & Related papers  (2021-10-12T17:21:41Z)
• Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
  Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters. We show that our method improves both accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2021-09-27T01:09:08Z)
• Predicting with Confidence on Unseen Distributions [90.68414180153897]
  We connect domain adaptation and predictive uncertainty literature to predict model accuracy on challenging unseen distributions. We find that the difference of confidences (DoC) of a classifier's predictions successfully estimates the classifier's performance change over a variety of shifts. We specifically investigate the distinction between synthetic and natural distribution shifts and observe that despite its simplicity DoC consistently outperforms other quantifications of distributional difference.
  arXiv  Detail & Related papers  (2021-07-07T15:50:18Z)
• Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting [98.57851612518758]
  Probabilistic time series forecasting involves estimating the distribution of future based on its history. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks. We show in experiments that our model produces accurate and sharp probabilistic forecasts.
  arXiv  Detail & Related papers  (2021-01-31T06:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.