Stability Analysis of Physics-Informed Neural Networks for Stiff Linear Differential Equations

• URL: http://arxiv.org/abs/2408.15393v1
• Date: Tue, 27 Aug 2024 20:33:16 GMT
• Title: Stability Analysis of Physics-Informed Neural Networks for Stiff Linear Differential Equations
• Authors: Gianluca Fabiani, Erik Bollt, Constantinos Siettos, Athanasios N. Yannacopoulos,
• Abstract summary: We present a stability analysis of Physics-Informed Neural Networks (PINNs) We consider systems of linear ODEs, and linear parabolic PDEs, for the numerical solution of (stiff) linear differential equations. We show that the proposed PINNs outperform the above traditional schemes, in both numerical approximation accuracy and importantly computational cost, for a wide range of step sizes.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a stability analysis of Physics-Informed Neural Networks (PINNs) coupled with random projections, for the numerical solution of (stiff) linear differential equations. For our analysis, we consider systems of linear ODEs, and linear parabolic PDEs. We prove that properly designed PINNs offer consistent and asymptotically stable numerical schemes, thus convergent schemes. In particular, we prove that multi-collocation random projection PINNs guarantee asymptotic stability for very high stiffness and that single-collocation PINNs are $A$-stable. To assess the performance of the PINNs in terms of both numerical approximation accuracy and computational cost, we compare it with other implicit schemes and in particular backward Euler, the midpoint, trapezoidal (Crank-Nikolson), the 2-stage Gauss scheme and the 2 and 3 stages Radau schemes. We show that the proposed PINNs outperform the above traditional schemes, in both numerical approximation accuracy and importantly computational cost, for a wide range of step sizes.

Related papers

• Physics-informed neural networks for high-dimensional solutions and snaking bifurcations in nonlinear lattices [0.0]
  This paper introduces a framework based on physics-informed neural networks (PINNs) for addressing key challenges in nonlinear lattices.<n>We first employ PINNs to approximate solutions of nonlinear systems arising from lattice models, using the Levenberg-Marquardt algorithm.<n>We then extend the method by coupling PINNs with a continuation approach to compute snaking bifurcation diagrams.<n>For linear stability analysis, we adapt PINNs to compute eigenvectors, introducing output constraints to enforce positivity, in line with Sturm-Liouville theory.
  arXiv  Detail & Related papers  (2025-07-13T20:41:55Z)
• High precision PINNs in unbounded domains: application to singularity formulation in PDEs [83.50980325611066]
  We study the choices of neural network ansatz, sampling strategy, and optimization algorithm.<n>For 1D Burgers equation, our framework can lead to a solution with very high precision.<n>For the 2D Boussinesq equation, we obtain a solution whose loss is $4$ digits smaller than that obtained in citewang2023asymptotic with fewer training steps.
  arXiv  Detail & Related papers  (2025-06-24T02:01:44Z)
• PINNs Algorithmic Framework for Simulation of Nonlinear Burgers' Type Models [1.124958340749622]
  A physics-informed neural networks (PINNs) based algorithm is used for simulation of nonlinear 1D and 2D Burgers' type models.<n>Results demonstrate that PINNs may faithfully replicate nonlinear PDE solutions and offer competitive performance in terms of inaccuracy and flexibility.
  arXiv  Detail & Related papers  (2025-06-15T17:39:15Z)
• Physics-informed Neural Networks for Functional Differential Equations: Cylindrical Approximation and Its Convergence Guarantees [7.366405857677226]
  We propose the first learning scheme for functional differential equations (FDEs) FDEs play a fundamental role in physics, mathematics, and optimal control. numerical approximations of FDEs have been developed, but they often oversimplify the solutions.
  arXiv  Detail & Related papers  (2024-10-23T06:16:35Z)
• General-Kindred Physics-Informed Neural Network to the Solutions of Singularly Perturbed Differential Equations [11.121415128908566]
  We propose the General-Kindred Physics-Informed Neural Network (GKPINN) for solving Singular Perturbation Differential Equations (SPDEs) This approach utilizes prior knowledge of the boundary layer from the equation and establishes a novel network to assist PINN in approxing the boundary layer. The research findings underscore the exceptional performance of our novel approach, GKPINN, which delivers a remarkable enhancement in reducing the $L$ error by two to four orders of magnitude compared to the established PINN methodology.
  arXiv  Detail & Related papers  (2024-08-27T02:03:22Z)
• Neural networks for bifurcation and linear stability analysis of steady states in partial differential equations [0.0]
  A neural network is proposed to construct bifurcation diagrams from parameterized nonlinear PDEs.<n>A neural network approach is also presented for solving eigenvalue problems to analyze solution linear stability.
  arXiv  Detail & Related papers  (2024-07-29T05:05:13Z)
• Nonlinear Discrete-Time Observers with Physics-Informed Neural Networks [0.0]
  We use Physics-Informed Neural Networks (PINNs) to solve the discrete-time nonlinear observer state estimation problem. The proposed PINN approach aims at learning a nonlinear state transformation map by solving a system of inhomogeneous functional equations.
  arXiv  Detail & Related papers  (2024-02-19T18:47:56Z)
• Learning solutions of parametric Navier-Stokes with physics-informed neural networks [0.3989223013441816]
  We leverageformed-Informed Neural Networks (PINs) to learn solution functions of parametric Navier-Stokes equations (NSE) We consider the parameter(s) of interest as inputs of PINs along with coordinates, and train PINs on numerical solutions of parametric-PDES for instances of the parameters. We show that our proposed approach results in optimizing PINN models that learn the solution functions while making sure that flow predictions are in line with conservational laws of mass and momentum.
  arXiv  Detail & Related papers  (2024-02-05T16:19:53Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• Identifiability and Asymptotics in Learning Homogeneous Linear ODE Systems from Discrete Observations [114.17826109037048]
  Ordinary Differential Equations (ODEs) have recently gained a lot of attention in machine learning. theoretical aspects, e.g., identifiability and properties of statistical estimation are still obscure. This paper derives a sufficient condition for the identifiability of homogeneous linear ODE systems from a sequence of equally-spaced error-free observations sampled from a single trajectory.
  arXiv  Detail & Related papers  (2022-10-12T06:46:38Z)
• Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions [68.8204255655161]
  We develop a physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. Localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. We propose criteria for weights in the loss function that produce a more accurate PINN solution than those obtained with the weights selected via other methods.
  arXiv  Detail & Related papers  (2022-08-18T05:00:24Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• Neural Operator with Regularity Structure for Modeling Dynamics Driven by SPDEs [70.51212431290611]
  Partial differential equations (SPDEs) are significant tools for modeling dynamics in many areas including atmospheric sciences and physics. We propose the Neural Operator with Regularity Structure (NORS) which incorporates the feature vectors for modeling dynamics driven by SPDEs. We conduct experiments on various of SPDEs including the dynamic Phi41 model and the 2d Navier-Stokes equation.
  arXiv  Detail & Related papers  (2022-04-13T08:53:41Z)
• Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs [0.0]
  We address a physics-informed neural network based on random projections for the numerical solution of IVPs of nonlinear ODEs in linear-implicit form and index-1 DAEs. Based on previous works on random projections, we prove the approximation capability of the scheme for ODEs in the canonical form and index-1 DAEs in the semiexplicit form.
  arXiv  Detail & Related papers  (2022-03-10T12:34:46Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• dNNsolve: an efficient NN-based PDE solver [62.997667081978825]
  We introduce dNNsolve, that makes use of dual Neural Networks to solve ODEs/PDEs. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions.
  arXiv  Detail & Related papers  (2021-03-15T19:14:41Z)
• Physics-Informed Neural Network Method for Solving One-Dimensional Advection Equation Using PyTorch [0.0]
  PINNs approach allows training neural networks while respecting the PDEs as a strong constraint in the optimization. In standard small-scale circulation simulations, it is shown that the conventional approach incorporates a pseudo diffusive effect that is almost as large as the effect of the turbulent diffusion model. Of all the schemes tested, only the PINNs approximation accurately predicted the outcome.
  arXiv  Detail & Related papers  (2021-03-15T05:39:17Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.