Learning the Delay Using Neural Delay Differential Equations
- URL: http://arxiv.org/abs/2304.01329v2
- Date: Wed, 14 Jun 2023 16:51:05 GMT
- Title: Learning the Delay Using Neural Delay Differential Equations
- Authors: Maria Oprea and Mark Walth and Robert Stephany and Gabriella Torres
Nothaft and Arnaldo Rodriguez-Gonzalez and William Clark
- Abstract summary: We develop a continuous time neural network approach based on Delay Differential Equations (DDEs)
Our model uses the adjoint sensitivity method to learn the model parameters and delay directly from data.
We conclude our discussion with potential future directions and applications.
- Score: 0.5505013339790825
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The intersection of machine learning and dynamical systems has generated
considerable interest recently. Neural Ordinary Differential Equations (NODEs)
represent a rich overlap between these fields. In this paper, we develop a
continuous time neural network approach based on Delay Differential Equations
(DDEs). Our model uses the adjoint sensitivity method to learn the model
parameters and delay directly from data. Our approach is inspired by that of
NODEs and extends earlier neural DDE models, which have assumed that the value
of the delay is known a priori. We perform a sensitivity analysis on our
proposed approach and demonstrate its ability to learn DDE parameters from
benchmark systems. We conclude our discussion with potential future directions
and applications.
Related papers
- Parametric Learning of Time-Advancement Operators for Unstable Flame
Evolution [0.0]
This study investigates the application of machine learning to learn time-advancement operators for parametric partial differential equations (PDEs)
Our focus is on extending existing operator learning methods to handle additional inputs representing PDE parameters.
The goal is to create a unified learning approach that accurately predicts short-term solutions and provides robust long-term statistics.
arXiv Detail & Related papers (2024-02-14T18:12:42Z) - Neural Delay Differential Equations: System Reconstruction and Image
Classification [14.59919398960571]
We propose a new class of continuous-depth neural networks with delay, named Neural Delay Differential Equations (NDDEs)
Compared to NODEs, NDDEs have a stronger capacity of nonlinear representations.
We achieve lower loss and higher accuracy not only for the data produced synthetically but also for the CIFAR10, a well-known image dataset.
arXiv Detail & Related papers (2023-04-11T16:09:28Z) - Neural Basis Functions for Accelerating Solutions to High Mach Euler
Equations [63.8376359764052]
We propose an approach to solving partial differential equations (PDEs) using a set of neural networks.
We regress a set of neural networks onto a reduced order Proper Orthogonal Decomposition (POD) basis.
These networks are then used in combination with a branch network that ingests the parameters of the prescribed PDE to compute a reduced order approximation to the PDE.
arXiv Detail & Related papers (2022-08-02T18:27:13Z) - Learning Time Delay Systems with Neural Ordinary Differential Equations [0.0]
A neural network with trainable delays is used to approximate a delay differential equation.
An example on learning the dynamics of the Mackey-Glass equation using data from chaotic behavior is given.
arXiv Detail & Related papers (2022-06-28T20:59:44Z) - Neural Laplace: Learning diverse classes of differential equations in
the Laplace domain [86.52703093858631]
We propose a unified framework for learning diverse classes of differential equations (DEs) including all the aforementioned ones.
Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials.
In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs.
arXiv Detail & Related papers (2022-06-10T02:14:59Z) - Incorporating NODE with Pre-trained Neural Differential Operator for
Learning Dynamics [73.77459272878025]
We propose to enhance the supervised signal in learning dynamics by pre-training a neural differential operator (NDO)
NDO is pre-trained on a class of symbolic functions, and it learns the mapping between the trajectory samples of these functions to their derivatives.
We provide theoretical guarantee on that the output of NDO can well approximate the ground truth derivatives by proper tuning the complexity of the library.
arXiv Detail & Related papers (2021-06-08T08:04:47Z) - Neural ODE Processes [64.10282200111983]
We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs.
We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
arXiv Detail & Related papers (2021-03-23T09:32:06Z) - Neural Delay Differential Equations [9.077775405204347]
We propose a new class of continuous-depth neural networks with delay, named as Neural Delay Differential Equations (NDDEs)
For computing the corresponding gradients, we use the adjoint sensitivity method to obtain the delayed dynamics of the adjoint.
Our results reveal that appropriately articulating the elements of dynamical systems into the network design is truly beneficial to promoting the network performance.
arXiv Detail & Related papers (2021-02-22T06:53:51Z) - Learning continuous-time PDEs from sparse data with graph neural
networks [10.259254824702555]
We propose a continuous-time differential model for dynamical systems whose governing equations are parameterized by message passing graph neural networks.
We demonstrate the model's ability to work with unstructured grids, arbitrary time steps, and noisy observations.
We compare our method with existing approaches on several well-known physical systems that involve first and higher-order PDEs with state-of-the-art predictive performance.
arXiv Detail & Related papers (2020-06-16T07:15:40Z) - Liquid Time-constant Networks [117.57116214802504]
We introduce a new class of time-continuous recurrent neural network models.
Instead of declaring a learning system's dynamics by implicit nonlinearities, we construct networks of linear first-order dynamical systems.
These neural networks exhibit stable and bounded behavior, yield superior expressivity within the family of neural ordinary differential equations.
arXiv Detail & Related papers (2020-06-08T09:53:35Z) - Time Dependence in Non-Autonomous Neural ODEs [74.78386661760662]
We propose a novel family of Neural ODEs with time-varying weights.
We outperform previous Neural ODE variants in both speed and representational capacity.
arXiv Detail & Related papers (2020-05-05T01:41:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.