Characteristic Neural Ordinary Differential Equations

• URL: http://arxiv.org/abs/2111.13207v1
• Date: Thu, 25 Nov 2021 18:25:09 GMT
• Title: Characteristic Neural Ordinary Differential Equations
• Authors: Xingzi Xu, Ali Hasan, Khalil Elkhalil, Jie Ding, Vahid Tarokh
• Abstract summary: We propose a framework for extending Neural Ordinary Differential Equations (NODEs) beyond ODEs. While NODEs model the evolution of the latent state as the solution to an ODE, the proposed C-NODE models the evolution of the latent state as the solution of a family of first-order quasi-linear partial differential equations (PDE) We prove that the C-NODE framework extends the classical NODE by exhibiting functions that cannot be represented by NODEs but are representable by C-NODEs.
• Score: 30.20663139358168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose Characteristic Neural Ordinary Differential Equations (C-NODEs), a framework for extending Neural Ordinary Differential Equations (NODEs) beyond ODEs. While NODEs model the evolution of the latent state as the solution to an ODE, the proposed C-NODE models the evolution of the latent state as the solution of a family of first-order quasi-linear partial differential equations (PDE) on their characteristics, defined as curves along which the PDEs reduce to ODEs. The reduction, in turn, allows the application of the standard frameworks for solving ODEs to PDE settings. Additionally, the proposed framework can be cast as an extension of existing NODE architectures, thereby allowing the use of existing black-box ODE solvers. We prove that the C-NODE framework extends the classical NODE by exhibiting functions that cannot be represented by NODEs but are representable by C-NODEs. We further investigate the efficacy of the C-NODE framework by demonstrating its performance in many synthetic and real data scenarios. Empirical results demonstrate the improvements provided by the proposed method for CIFAR-10, SVHN, and MNIST datasets under a similar computational budget as the existing NODE methods.

Related papers

• Unisolver: PDE-Conditional Transformers Are Universal PDE Solvers [55.0876373185983]
  We present the Universal PDE solver (Unisolver) capable of solving a wide scope of PDEs. Our key finding is that a PDE solution is fundamentally under the control of a series of PDE components. Unisolver achieves consistent state-of-the-art results on three challenging large-scale benchmarks.
  arXiv  Detail & Related papers  (2024-05-27T15:34:35Z)
• Deep Equilibrium Based Neural Operators for Steady-State PDEs [100.88355782126098]
  We study the benefits of weight-tied neural network architectures for steady-state PDEs. We propose FNO-DEQ, a deep equilibrium variant of the FNO architecture that directly solves for the solution of a steady-state PDE.
  arXiv  Detail & Related papers  (2023-11-30T22:34:57Z)
• Reduced-order modeling for parameterized PDEs via implicit neural representations [4.135710717238787]
  We present a new data-driven reduced-order modeling approach to efficiently solve parametrized partial differential equations (PDEs) The proposed framework encodes PDE and utilizes a parametrized neural ODE (PNODE) to learn latent dynamics characterized by multiple PDE parameters. We evaluate the proposed method at a large Reynolds number and obtain up to speedup of O(103) and 1% relative error to the ground truth values.
  arXiv  Detail & Related papers  (2023-11-28T01:35:06Z)
• GNRK: Graph Neural Runge-Kutta method for solving partial differential equations [0.0]
  This study introduces a novel approach called Graph Neural Runge-Kutta (GNRK) GNRK integrates graph neural network modules with a recurrent structure inspired by the classical solvers. It demonstrates the capability to address general PDEs, irrespective of initial conditions or PDE coefficients.
  arXiv  Detail & Related papers  (2023-10-01T08:52:46Z)
• A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks [52.5899851000193]
  We develop an ODE based IVP solver which prevents the network from getting ill-conditioned and runs in time linear in the number of parameters. We show that current methods based on this approach suffer from two key issues. First, following the ODE produces an uncontrolled growth in the conditioning of the problem, ultimately leading to unacceptably large numerical errors.
  arXiv  Detail & Related papers  (2023-04-28T17:28:18Z)
• Koopman neural operator as a mesh-free solver of non-linear partial differential equations [15.410070455154138]
  We propose the Koopman neural operator (KNO), a new neural operator, to overcome these challenges. By approximating the Koopman operator, an infinite-dimensional operator governing all possible observations of the dynamic system, we can equivalently learn the solution of a non-linear PDE family. The KNO exhibits notable advantages compared with previous state-of-the-art models.
  arXiv  Detail & Related papers  (2023-01-24T14:10:15Z)
• Learning differentiable solvers for systems with hard constraints [48.54197776363251]
  We introduce a practical method to enforce partial differential equation (PDE) constraints for functions defined by neural networks (NNs) We develop a differentiable PDE-constrained layer that can be incorporated into any NN architecture. Our results show that incorporating hard constraints directly into the NN architecture achieves much lower test error when compared to training on an unconstrained objective.
  arXiv  Detail & Related papers  (2022-07-18T15:11:43Z)
• dNNsolve: an efficient NN-based PDE solver [62.997667081978825]
  We introduce dNNsolve, that makes use of dual Neural Networks to solve ODEs/PDEs. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions.
  arXiv  Detail & Related papers  (2021-03-15T19:14:41Z)
• Meta-Solver for Neural Ordinary Differential Equations [77.8918415523446]
  We investigate how the variability in solvers' space can improve neural ODEs performance. We show that the right choice of solver parameterization can significantly affect neural ODEs models in terms of robustness to adversarial attacks.
  arXiv  Detail & Related papers  (2021-03-15T17:26:34Z)
• Neural Delay Differential Equations [9.077775405204347]
  We propose a new class of continuous-depth neural networks with delay, named as Neural Delay Differential Equations (NDDEs) For computing the corresponding gradients, we use the adjoint sensitivity method to obtain the delayed dynamics of the adjoint. Our results reveal that appropriately articulating the elements of dynamical systems into the network design is truly beneficial to promoting the network performance.
  arXiv  Detail & Related papers  (2021-02-22T06:53:51Z)
• Neural SDEs as Infinite-Dimensional GANs [18.07683058213448]
  We show that the current classical approach to fitting SDEs may be approached as a special case of (Wasserstein) GANs. We obtain Neural SDEs as (in modern machine learning parlance) continuous-time generative time series models.
  arXiv  Detail & Related papers  (2021-02-06T19:59:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.