Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering

• URL: http://arxiv.org/abs/2006.04727v4
• Date: Fri, 16 Apr 2021 12:54:38 GMT
• Title: Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering
• Authors: Calypso Herrera, Florian Krach, Josef Teichmann
• Abstract summary: We introduce the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn, continuously in time. We show that our model converges to the $L2$-optimal online prediction. We experimentally show that our model outperforms the baselines in more complex learning tasks.
• Score: 6.445605125467574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Combinations of neural ODEs with recurrent neural networks (RNN), like GRU-ODE-Bayes or ODE-RNN are well suited to model irregularly observed time series. While those models outperform existing discrete-time approaches, no theoretical guarantees for their predictive capabilities are available. Assuming that the irregularly-sampled time series data originates from a continuous stochastic process, the $L^2$-optimal online prediction is the conditional expectation given the currently available information. We introduce the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn, continuously in time, the conditional expectation of a stochastic process. Our approach models the conditional expectation between two observations with a neural ODE and jumps whenever a new observation is made. We define a novel training framework, which allows us to prove theoretical guarantees for the first time. In particular, we show that the output of our model converges to the $L^2$-optimal prediction. This can be interpreted as solution to a special filtering problem. We provide experiments showing that the theoretical results also hold empirically. Moreover, we experimentally show that our model outperforms the baselines in more complex learning tasks and give comparisons on real-world datasets.

Related papers

• Deep Limit Model-free Prediction in Regression [0.0]
  We provide a Model-free approach based on Deep Neural Network (DNN) to accomplish point prediction and prediction interval under a general regression setting. Our method is more stable and accurate compared to other DNN-based counterparts, especially for optimal point predictions.
  arXiv  Detail & Related papers  (2024-08-18T16:37:53Z)
• Learning Chaotic Systems and Long-Term Predictions with Neural Jump ODEs [4.204990010424083]
  Pathdependent Neural Jump ODE (PDNJ-ODE) is a model for online prediction of generic processes with irregular (in time) and potentially incomplete (with respect to coordinates) observations. In this work we enhance the model with two novel ideas, which independently of each other improve the performance of our modelling setup. The same enhancements can be used to provably enable the PDNJ-ODE to learn long-term predictions for general datasets, where the standard model fails.
  arXiv  Detail & Related papers  (2024-07-26T15:18:29Z)
• Foundational Inference Models for Dynamical Systems [5.549794481031468]
  We offer a fresh perspective on the classical problem of imputing missing time series data, whose underlying dynamics are assumed to be determined by ODEs. We propose a novel supervised learning framework for zero-shot time series imputation, through parametric functions satisfying some (hidden) ODEs. We empirically demonstrate that one and the same (pretrained) recognition model can perform zero-shot imputation across 63 distinct time series with missing values.
  arXiv  Detail & Related papers  (2024-02-12T11:48:54Z)
• Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs [50.25683648762602]
  We introduce Koopman VAE, a new generative framework that is based on a novel design for the model prior. Inspired by Koopman theory, we represent the latent conditional prior dynamics using a linear map. KoVAE outperforms state-of-the-art GAN and VAE methods across several challenging synthetic and real-world time series generation benchmarks.
  arXiv  Detail & Related papers  (2023-10-04T07:14:43Z)
• Neural Differential Recurrent Neural Network with Adaptive Time Steps [11.999568208578799]
  We propose an RNN-based model, called RNN-ODE-Adap, that uses a neural ODE to represent the time development of the hidden states. We adaptively select time steps based on the steepness of changes of the data over time so as to train the model more efficiently for the "spike-like" time series.
  arXiv  Detail & Related papers  (2023-06-02T16:46:47Z)
• Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unit [56.801856519460465]
  Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations. We demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting.
  arXiv  Detail & Related papers  (2023-04-14T09:39:06Z)
• Artificial neural networks and time series of counts: A class of nonlinear INGARCH models [0.0]
  It is shown how INGARCH models can be combined with artificial neural network (ANN) response functions to obtain a class of nonlinear INGARCH models. The ANN framework allows for the interpretation of many existing INGARCH models as a degenerate version of a corresponding neural model. The empirical analysis of time series of bounded and unbounded counts reveals that the neural INGARCH models are able to outperform reasonable degenerate competitor models in terms of the information loss.
  arXiv  Detail & Related papers  (2023-04-03T14:26:16Z)
• On the balance between the training time and interpretability of neural ODE for time series modelling [77.34726150561087]
  The paper shows that modern neural ODE cannot be reduced to simpler models for time-series modelling applications. The complexity of neural ODE is compared to or exceeds the conventional time-series modelling tools. We propose a new view on time-series modelling using combined neural networks and an ODE system approach.
  arXiv  Detail & Related papers  (2022-06-07T13:49:40Z)
• Discovering Invariant Rationales for Graph Neural Networks [104.61908788639052]
  Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features. We propose a new strategy of discovering invariant rationale (DIR) to construct intrinsically interpretable GNNs.
  arXiv  Detail & Related papers  (2022-01-30T16:43:40Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.