Comprehensive Review of Neural Differential Equations for Time Series Analysis

• URL: http://arxiv.org/abs/2502.09885v1
• Date: Fri, 14 Feb 2025 03:21:04 GMT
• Title: Comprehensive Review of Neural Differential Equations for Time Series Analysis
• Authors: YongKyung Oh, Seungsu Kam, Jonghun Lee, Dong-Young Lim, Sungil Kim, Alex Bui,
• Abstract summary: This paper presents a comprehensive review of NDE-based methods for time series analysis. NDEs represent a paradigm shift by combining the flexibility of neural networks with the mathematical rigor of differential equations. We provide a detailed discussion of their mathematical formulations, numerical methods, and applications, highlighting their ability to model continuous-time dynamics.
• Score: 2.9687381456164004
• License:
• Abstract: Time series modeling and analysis has become critical in various domains. Conventional methods such as RNNs and Transformers, while effective for discrete-time and regularly sampled data, face significant challenges in capturing the continuous dynamics and irregular sampling patterns inherent in real-world scenarios. Neural Differential Equations (NDEs) represent a paradigm shift by combining the flexibility of neural networks with the mathematical rigor of differential equations. This paper presents a comprehensive review of NDE-based methods for time series analysis, including neural ordinary differential equations, neural controlled differential equations, and neural stochastic differential equations. We provide a detailed discussion of their mathematical formulations, numerical methods, and applications, highlighting their ability to model continuous-time dynamics. Furthermore, we address key challenges and future research directions. This survey serves as a foundation for researchers and practitioners seeking to leverage NDEs for advanced time series analysis.

Related papers

• Individualized Dosing Dynamics via Neural Eigen Decomposition [51.62933814971523]
  We introduce the Neural Eigen Differential Equation algorithm (NESDE) NESDE provides individualized modeling, tunable generalization to new treatment policies, and fast, continuous, closed-form prediction. We demonstrate the robustness of NESDE in both synthetic and real medical problems, and use the learned dynamics to publish simulated medical gym environments.
  arXiv  Detail & Related papers  (2023-06-24T17:01:51Z)
• Capturing dynamical correlations using implicit neural representations [85.66456606776552]
  We develop an artificial intelligence framework which combines a neural network trained to mimic simulated data from a model Hamiltonian with automatic differentiation to recover unknown parameters from experimental data. In doing so, we illustrate the ability to build and train a differentiable model only once, which then can be applied in real-time to multi-dimensional scattering data.
  arXiv  Detail & Related papers  (2023-04-08T07:55:36Z)
• Anamnesic Neural Differential Equations with Orthogonal Polynomial Projections [6.345523830122166]
  We propose PolyODE, a formulation that enforces long-range memory and preserves a global representation of the underlying dynamical system. Our construction is backed by favourable theoretical guarantees and we demonstrate that it outperforms previous works in the reconstruction of past and future data.
  arXiv  Detail & Related papers  (2023-03-03T10:49:09Z)
• Generalized Neural Closure Models with Interpretability [28.269731698116257]
  We develop a novel and versatile methodology of unified neural partial delay differential equations. We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations. We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models.
  arXiv  Detail & Related papers  (2023-01-15T21:57:43Z)
• Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations [84.42837346400151]
  Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare. Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions. We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
  arXiv  Detail & Related papers  (2022-06-16T17:15:15Z)
• Neural Laplace: Learning diverse classes of differential equations in the Laplace domain [86.52703093858631]
  We propose a unified framework for learning diverse classes of differential equations (DEs) including all the aforementioned ones. Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials. In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs.
  arXiv  Detail & Related papers  (2022-06-10T02:14:59Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• On Neural Differential Equations [13.503274710499971]
  In particular, neural differential equations (NDEs) demonstrate that neural networks and differential equations are two sides of the same coin. NDEs are suitable for tackling generative problems, dynamical systems, and time series. NDEs offer high-capacity function approximation, strong priors on model space, the ability to handle irregular data, memory efficiency, and a wealth of available theory on both sides.
  arXiv  Detail & Related papers  (2022-02-04T23:32:29Z)
• Consistency of mechanistic causal discovery in continuous-time using Neural ODEs [85.7910042199734]
  We consider causal discovery in continuous-time for the study of dynamical systems. We propose a causal discovery algorithm based on penalized Neural ODEs.
  arXiv  Detail & Related papers  (2021-05-06T08:48:02Z)
• Learning Continuous-Time Dynamics by Stochastic Differential Networks [32.63114111531396]
  We propose a flexible continuous-time recurrent neural network named Variational Differential Networks (VSDN) VSDN embeds the complicated dynamics of the sporadic time series by neural Differential Equations (SDE) We show that VSDNs outperform state-of-the-art continuous-time deep learning models and achieve remarkable performance on prediction and tasks for sporadic time series.
  arXiv  Detail & Related papers  (2020-06-11T01:40:34Z)
• Machine Learning from a Continuous Viewpoint [12.865834066050427]
  We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations. We discuss how the issues of generalization error and implicit regularization can be studied under this framework.
  arXiv  Detail & Related papers  (2019-12-30T01:38:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.