ControlSynth Neural ODEs: Modeling Dynamical Systems with Guaranteed Convergence

• URL: http://arxiv.org/abs/2411.02292v1
• Date: Mon, 04 Nov 2024 17:20:42 GMT
• Title: ControlSynth Neural ODEs: Modeling Dynamical Systems with Guaranteed Convergence
• Authors: Wenjie Mei, Dongzhe Zheng, Shihua Li,
• Abstract summary: Neural ODEs (NODEs) are continuous-time neural networks (NNs) that can process data without the limitation of time intervals. We show that despite their highly nonlinear nature, convergence can be guaranteed via tractable linear inequalities. In the composition of CSODEs, we introduce an extra control term for learning the potential simultaneous capture of dynamics at different scales.
• Score: 1.1720409777196028
• License:
• Abstract: Neural ODEs (NODEs) are continuous-time neural networks (NNs) that can process data without the limitation of time intervals. They have advantages in learning and understanding the evolution of complex real dynamics. Many previous works have focused on NODEs in concise forms, while numerous physical systems taking straightforward forms, in fact, belong to their more complex quasi-classes, thus appealing to a class of general NODEs with high scalability and flexibility to model those systems. This, however, may result in intricate nonlinear properties. In this paper, we introduce ControlSynth Neural ODEs (CSODEs). We show that despite their highly nonlinear nature, convergence can be guaranteed via tractable linear inequalities. In the composition of CSODEs, we introduce an extra control term for learning the potential simultaneous capture of dynamics at different scales, which could be particularly useful for partial differential equation-formulated systems. Finally, we compare several representative NNs with CSODEs on important physical dynamics under the inductive biases of CSODEs, and illustrate that CSODEs have better learning and predictive abilities in these settings.

Related papers

• KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics [0.0]
  Kolmogorov-Arnold networks (KANs) are an alternative to multi-layer perceptrons (MLPs) This work applies KANs as the backbone of a neural ordinary differential equation (ODE) framework.
  arXiv  Detail & Related papers  (2024-07-05T00:38:49Z)
• Neural Fractional Differential Equations [2.812395851874055]
  Fractional Differential Equations (FDEs) are essential tools for modelling complex systems in science and engineering. We propose the Neural FDE, a novel deep neural network architecture that adjusts a FDE to the dynamics of data.
  arXiv  Detail & Related papers  (2024-03-05T07:45:29Z)
• Learning Neural Constitutive Laws From Motion Observations for Generalizable PDE Dynamics [97.38308257547186]
  Many NN approaches learn an end-to-end model that implicitly models both the governing PDE and material models. We argue that the governing PDEs are often well-known and should be explicitly enforced rather than learned. We introduce a new framework termed "Neural Constitutive Laws" (NCLaw) which utilizes a network architecture that strictly guarantees standard priors.
  arXiv  Detail & Related papers  (2023-04-27T17:42:24Z)
• Neural Delay Differential Equations: System Reconstruction and Image Classification [14.59919398960571]
  We propose a new class of continuous-depth neural networks with delay, named Neural Delay Differential Equations (NDDEs) Compared to NODEs, NDDEs have a stronger capacity of nonlinear representations. We achieve lower loss and higher accuracy not only for the data produced synthetically but also for the CIFAR10, a well-known image dataset.
  arXiv  Detail & Related papers  (2023-04-11T16:09:28Z)
• Decomposed Linear Dynamical Systems (dLDS) for learning the latent components of neural dynamics [6.829711787905569]
  We propose a new decomposed dynamical system model that represents complex non-stationary and nonlinear dynamics of time series data. Our model is trained through a dictionary learning procedure, where we leverage recent results in tracking sparse vectors over time. In both continuous-time and discrete-time instructional examples we demonstrate that our model can well approximate the original system.
  arXiv  Detail & Related papers  (2022-06-07T02:25:38Z)
• Stabilized Neural Ordinary Differential Equations for Long-Time Forecasting of Dynamical Systems [1.001737665513683]
  We present a data-driven modeling method that accurately captures shocks and chaotic dynamics. We learn the right-hand-side (SRH) of an ODE by adding the outputs of two NN together where one learns a linear term and the other a nonlinear term. Specifically, we implement this by training a sparse linear convolutional NN to learn the linear term and a dense fully-connected nonlinear NN to learn the nonlinear term.
  arXiv  Detail & Related papers  (2022-03-29T16:10:34Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Neural ODE Processes [64.10282200111983]
  We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs. We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
  arXiv  Detail & Related papers  (2021-03-23T09:32:06Z)
• Continuous-in-Depth Neural Networks [107.47887213490134]
  We first show that ResNets fail to be meaningful dynamical in this richer sense. We then demonstrate that neural network models can learn to represent continuous dynamical systems. We introduce ContinuousNet as a continuous-in-depth generalization of ResNet architectures.
  arXiv  Detail & Related papers  (2020-08-05T22:54:09Z)
• On Second Order Behaviour in Augmented Neural ODEs [69.8070643951126]
  We consider Second Order Neural ODEs (SONODEs) We show how the adjoint sensitivity method can be extended to SONODEs. We extend the theoretical understanding of the broader class of Augmented NODEs (ANODEs)
  arXiv  Detail & Related papers  (2020-06-12T14:25:31Z)
• Learning Stable Deep Dynamics Models [91.90131512825504]
  We propose an approach for learning dynamical systems that are guaranteed to be stable over the entire state space. We show that such learning systems are able to model simple dynamical systems and can be combined with additional deep generative models to learn complex dynamics.
  arXiv  Detail & Related papers  (2020-01-17T00:04:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.