Stiff Neural Ordinary Differential Equations

• URL: http://arxiv.org/abs/2103.15341v1
• Date: Mon, 29 Mar 2021 05:24:56 GMT
• Title: Stiff Neural Ordinary Differential Equations
• Authors: Suyong Kim, Weiqi Ji, Sili Deng, Christopher Rackauckas
• Abstract summary: We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODE. The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering, and the life sciences.

Related papers

• Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods [3.941173292703699]
  Stiff systems of ordinary differential equations (ODEs) are pervasive in many science and engineering fields. Standard neural ODE approaches struggle to learn them. This paper proposes an approach based on single-step implicit schemes to enable neural ODEs to handle stiffness.
  arXiv  Detail & Related papers  (2024-10-08T01:08:17Z)
• KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics [0.0]
  Kolmogorov-Arnold networks (KANs) are an alternative to multi-layer perceptrons (MLPs) This work applies KANs as the backbone of a neural ordinary differential equation (ODE) framework.
  arXiv  Detail & Related papers  (2024-07-05T00:38:49Z)
• Learning Governing Equations of Unobserved States in Dynamical Systems [0.0]
  We employ a hybrid neural ODE structure to learn governing equations of partially-observed dynamical systems. We demonstrate that the method is capable of successfully learning the true underlying governing equations of unobserved states within these systems.
  arXiv  Detail & Related papers  (2024-04-29T10:28:14Z)
• Experimental study of Neural ODE training with adaptive solver for dynamical systems modeling [72.84259710412293]
  Some ODE solvers called adaptive can adapt their evaluation strategy depending on the complexity of the problem at hand. This paper describes a simple set of experiments to show why adaptive solvers cannot be seamlessly leveraged as a black-box for dynamical systems modelling.
  arXiv  Detail & Related papers  (2022-11-13T17:48:04Z)
• Neural Laplace: Learning diverse classes of differential equations in the Laplace domain [86.52703093858631]
  We propose a unified framework for learning diverse classes of differential equations (DEs) including all the aforementioned ones. Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials. In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs.
  arXiv  Detail & Related papers  (2022-06-10T02:14:59Z)
• Physics Informed RNN-DCT Networks for Time-Dependent Partial Differential Equations [62.81701992551728]
  We present a physics-informed framework for solving time-dependent partial differential equations. Our model utilizes discrete cosine transforms to encode spatial and recurrent neural networks. We show experimental results on the Taylor-Green vortex solution to the Navier-Stokes equations.
  arXiv  Detail & Related papers  (2022-02-24T20:46:52Z)
• Constructing Neural Network-Based Models for Simulating Dynamical Systems [59.0861954179401]
  Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system. This paper provides a survey of the different ways to construct models of dynamical systems using neural networks. In addition to the basic overview, we review the related literature and outline the most significant challenges from numerical simulations that this modeling paradigm must overcome.
  arXiv  Detail & Related papers  (2021-11-02T10:51:42Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Accelerating Neural ODEs Using Model Order Reduction [0.0]
  We show that mathematical model order reduction methods can be used for compressing and accelerating Neural ODEs. We implement our novel compression method by developing Neural ODEs that integrate the necessary subspace-projection and operations as layers of the neural network.
  arXiv  Detail & Related papers  (2021-05-28T19:27:09Z)
• Neural ODE Processes [64.10282200111983]
  We introduce Neural ODE Processes (NDPs), a new class of processes determined by a distribution over Neural ODEs. We show that our model can successfully capture the dynamics of low-dimensional systems from just a few data-points.
  arXiv  Detail & Related papers  (2021-03-23T09:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.