Functional Latent Dynamics for Irregularly Sampled Time Series Forecasting

• URL: http://arxiv.org/abs/2405.03582v2
• Date: Thu, 03 Oct 2024 12:18:09 GMT
• Title: Functional Latent Dynamics for Irregularly Sampled Time Series Forecasting
• Authors: Christian Klötergens, Vijaya Krishna Yalavarthi, Maximilian Stubbemann, Lars Schmidt-Thieme,
• Abstract summary: Irregularly sampled time series with missing values are often observed in multiple real-world applications such as healthcare, climate and astronomy. We propose a family of models called Functional Latent Dynamics (FLD) Instead of solving the Ordinary Differential Equation (ODE), we use simple curves which exist at all time points to specify the continuous latent state in the model.
• Score: 5.359176539960004
• License:
• Abstract: Irregularly sampled time series with missing values are often observed in multiple real-world applications such as healthcare, climate and astronomy. They pose a significant challenge to standard deep learning models that operate only on fully observed and regularly sampled time series. In order to capture the continuous dynamics of the irregular time series, many models rely on solving an Ordinary Differential Equation (ODE) in the hidden state. These ODE-based models tend to perform slow and require large memory due to sequential operations and a complex ODE solver. As an alternative to complex ODE-based models, we propose a family of models called Functional Latent Dynamics (FLD). Instead of solving the ODE, we use simple curves which exist at all time points to specify the continuous latent state in the model. The coefficients of these curves are learned only from the observed values in the time series ignoring the missing values. Through extensive experiments, we demonstrate that FLD achieves better performance compared to the best ODE-based model while reducing the runtime and memory overhead. Specifically, FLD requires an order of magnitude less time to infer the forecasts compared to the best performing forecasting model.

Related papers

• Path-minimizing Latent ODEs for improved extrapolation and inference [0.0]
  Latent ODE models provide flexible descriptions of dynamic systems, but they can struggle with extrapolation and predicting complicated non-linear dynamics. In this paper we exploit this dichotomy by encouraging time-independent latent representations. By replacing the common variational penalty in latent space with an $ell$ penalty on the path length of each system, the models learn data representations that can easily be distinguished from those of systems with different configurations. This results in faster training, smaller models, more accurate and long-time extrapolation compared to the baseline ODE models with GRU, RNN, and LSTM/decoders on tests with
  arXiv  Detail & Related papers  (2024-10-11T15:50:01Z)
• Foundational Inference Models for Dynamical Systems [5.549794481031468]
  We offer a fresh perspective on the classical problem of imputing missing time series data, whose underlying dynamics are assumed to be determined by ODEs. We propose a novel supervised learning framework for zero-shot time series imputation, through parametric functions satisfying some (hidden) ODEs. We empirically demonstrate that one and the same (pretrained) recognition model can perform zero-shot imputation across 63 distinct time series with missing values.
  arXiv  Detail & Related papers  (2024-02-12T11:48:54Z)
• Deep Latent State Space Models for Time-Series Generation [68.45746489575032]
  We propose LS4, a generative model for sequences with latent variables evolving according to a state space ODE. Inspired by recent deep state space models (S4), we achieve speedups by leveraging a convolutional representation of LS4. We show that LS4 significantly outperforms previous continuous-time generative models in terms of marginal distribution, classification, and prediction scores on real-world datasets.
  arXiv  Detail & Related papers  (2022-12-24T15:17:42Z)
• SeqLink: A Robust Neural-ODE Architecture for Modelling Partially Observed Time Series [11.261457967759688]
  We introduce SeqLink, an innovative neural architecture designed to enhance the robustness of sequence representation. We demonstrate that SeqLink improves the modelling of intermittent time series, consistently outperforming state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-12-07T10:25:59Z)
• Discovering ordinary differential equations that govern time-series [65.07437364102931]
  We propose a transformer-based sequence-to-sequence model that recovers scalar autonomous ordinary differential equations (ODEs) in symbolic form from time-series data of a single observed solution of the ODE. Our method is efficiently scalable: after one-time pretraining on a large set of ODEs, we can infer the governing laws of a new observed solution in a few forward passes of the model.
  arXiv  Detail & Related papers  (2022-11-05T07:07:58Z)
• DynaConF: Dynamic Forecasting of Non-Stationary Time Series [4.286546152336783]
  We propose a new method to model non-stationary conditional distributions over time. We show that our model can adapt to non-stationary time series better than state-of-the-art deep learning solutions.
  arXiv  Detail & Related papers  (2022-09-17T21:40:02Z)
• On the balance between the training time and interpretability of neural ODE for time series modelling [77.34726150561087]
  The paper shows that modern neural ODE cannot be reduced to simpler models for time-series modelling applications. The complexity of neural ODE is compared to or exceeds the conventional time-series modelling tools. We propose a new view on time-series modelling using combined neural networks and an ODE system approach.
  arXiv  Detail & Related papers  (2022-06-07T13:49:40Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• STEER: Simple Temporal Regularization For Neural ODEs [80.80350769936383]
  We propose a new regularization technique: randomly sampling the end time of the ODE during training. The proposed regularization is simple to implement, has negligible overhead and is effective across a wide variety of tasks. We show through experiments on normalizing flows, time series models and image recognition that the proposed regularization can significantly decrease training time and even improve performance over baseline models.
  arXiv  Detail & Related papers  (2020-06-18T17:44:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.