Stabilized Neural Differential Equations for Learning Dynamics with
Explicit Constraints
- URL: http://arxiv.org/abs/2306.09739v3
- Date: Thu, 15 Feb 2024 16:47:31 GMT
- Title: Stabilized Neural Differential Equations for Learning Dynamics with
Explicit Constraints
- Authors: Alistair White, Niki Kilbertus, Maximilian Gelbrecht, Niklas Boers
- Abstract summary: We propose stabilized neural differential equations (SNDEs) to enforce arbitrary manifold constraints for neural differential equations.
Our approach is based on a stabilization term that, when added to the original dynamics, renders the constraint manifold provably stable.
Due to its simplicity, our method is compatible with all common neural differential equation (NDE) models and broadly applicable.
- Score: 4.656302602746229
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Many successful methods to learn dynamical systems from data have recently
been introduced. However, ensuring that the inferred dynamics preserve known
constraints, such as conservation laws or restrictions on the allowed system
states, remains challenging. We propose stabilized neural differential
equations (SNDEs), a method to enforce arbitrary manifold constraints for
neural differential equations. Our approach is based on a stabilization term
that, when added to the original dynamics, renders the constraint manifold
provably asymptotically stable. Due to its simplicity, our method is compatible
with all common neural differential equation (NDE) models and broadly
applicable. In extensive empirical evaluations, we demonstrate that SNDEs
outperform existing methods while broadening the types of constraints that can
be incorporated into NDE training.
Related papers
- Quantum Algorithms for Nonlinear Dynamics: Revisiting Carleman Linearization with No Dissipative Conditions [0.7373617024876725]
We explore the embedding of nonlinear dynamical systems into linear ordinary differential equations (ODEs) via the Carleman linearization method.
Our analysis extends these findings by exploring error bounds beyond the traditional dissipative condition.
We prove how this resonance condition leads to a linear convergence with respect to the truncation level $N$ in Carleman linearization.
arXiv Detail & Related papers (2024-05-21T12:09:34Z) - Neural State-Dependent Delay Differential Equations [0.0]
Discontinuities and delayed terms are encountered in the governing equations of a large class of problems ranging from physics to medicine.
We revisit the recently proposed Neural DDE by introducing Neural State-Dependent DDE (SDDDE), a general and flexible framework featuring multiple and state-dependent delays.
We show that our method is competitive and outperforms other continuous-class models on a wide variety of delayed dynamical systems.
arXiv Detail & Related papers (2023-06-26T09:35:56Z) - Generalized Neural Closure Models with Interpretability [28.269731698116257]
We develop a novel and versatile methodology of unified neural partial delay differential equations.
We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations.
We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models.
arXiv Detail & Related papers (2023-01-15T21:57:43Z) - Guaranteed Conservation of Momentum for Learning Particle-based Fluid
Dynamics [96.9177297872723]
We present a novel method for guaranteeing linear momentum in learned physics simulations.
We enforce conservation of momentum with a hard constraint, which we realize via antisymmetrical continuous convolutional layers.
In combination, the proposed method allows us to increase the physical accuracy of the learned simulator substantially.
arXiv Detail & Related papers (2022-10-12T09:12:59Z) - Neural Laplace: Learning diverse classes of differential equations in
the Laplace domain [86.52703093858631]
We propose a unified framework for learning diverse classes of differential equations (DEs) including all the aforementioned ones.
Instead of modelling the dynamics in the time domain, we model it in the Laplace domain, where the history-dependencies and discontinuities in time can be represented as summations of complex exponentials.
In the experiments, Neural Laplace shows superior performance in modelling and extrapolating the trajectories of diverse classes of DEs.
arXiv Detail & Related papers (2022-06-10T02:14:59Z) - Decimation technique for open quantum systems: a case study with
driven-dissipative bosonic chains [62.997667081978825]
Unavoidable coupling of quantum systems to external degrees of freedom leads to dissipative (non-unitary) dynamics.
We introduce a method to deal with these systems based on the calculation of (dissipative) lattice Green's function.
We illustrate the power of this method with several examples of driven-dissipative bosonic chains of increasing complexity.
arXiv Detail & Related papers (2022-02-15T19:00:09Z) - Message Passing Neural PDE Solvers [60.77761603258397]
We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators.
We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes.
We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
arXiv Detail & Related papers (2022-02-07T17:47:46Z) - Structure-Preserving Learning Using Gaussian Processes and Variational
Integrators [62.31425348954686]
We propose the combination of a variational integrator for the nominal dynamics of a mechanical system and learning residual dynamics with Gaussian process regression.
We extend our approach to systems with known kinematic constraints and provide formal bounds on the prediction uncertainty.
arXiv Detail & Related papers (2021-12-10T11:09:29Z) - Training Generative Adversarial Networks by Solving Ordinary
Differential Equations [54.23691425062034]
We study the continuous-time dynamics induced by GAN training.
From this perspective, we hypothesise that instabilities in training GANs arise from the integration error.
We experimentally verify that well-known ODE solvers (such as Runge-Kutta) can stabilise training.
arXiv Detail & Related papers (2020-10-28T15:23:49Z) - ImitationFlow: Learning Deep Stable Stochastic Dynamic Systems by
Normalizing Flows [29.310742141970394]
We introduce ImitationFlow, a novel Deep generative model that allows learning complex globally stable, nonlinear dynamics.
We show the effectiveness of our method with both standard datasets and a real robot experiment.
arXiv Detail & Related papers (2020-10-25T14:49:46Z) - Constrained Neural Ordinary Differential Equations with Stability
Guarantees [1.1086440815804224]
We show how to model discrete ordinary differential equations with algebraic nonlinearities as deep neural networks.
We derive the stability guarantees of the network layers based on the implicit constraints imposed on the weight's eigenvalues.
We demonstrate the prediction accuracy of learned neural ODEs evaluated on open-loop simulations.
arXiv Detail & Related papers (2020-04-22T22:07:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.