Replacing Automatic Differentiation by Sobolev Cubatures fastens Physics Informed Neural Nets and strengthens their Approximation Power

• URL: http://arxiv.org/abs/2211.15443v1
• Date: Wed, 23 Nov 2022 11:23:08 GMT
• Title: Replacing Automatic Differentiation by Sobolev Cubatures fastens Physics Informed Neural Nets and strengthens their Approximation Power
• Authors: Juan Esteban Suarez Cardona and Michael Hecht
• Abstract summary: We present a novel class of approximations for variational losses, being applicable for the training of physics-informed neural nets (PINNs) The loss computation rests on an extension of Gauss-Legendre cubatures, we term Sobolev cubatures, replacing automatic differentiation (A.D.)
• Score: 0.6091702876917279
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel class of approximations for variational losses, being applicable for the training of physics-informed neural nets (PINNs). The loss formulation reflects classic Sobolev space theory for partial differential equations and their weak formulations. The loss computation rests on an extension of Gauss-Legendre cubatures, we term Sobolev cubatures, replacing automatic differentiation (A.D.). We prove the runtime complexity of training the resulting Soblev-PINNs (SC-PINNs) to be less than required by PINNs relying on A.D. On top of one-to-two order of magnitude speed-up the SC-PINNs are demonstrated to achieve closer solution approximations for prominent forward and inverse PDE problems than established PINNs achieve.

Related papers

• Solving Poisson Equations using Neural Walk-on-Spheres [80.1675792181381]
  We propose Neural Walk-on-Spheres (NWoS), a novel neural PDE solver for the efficient solution of high-dimensional Poisson equations. We demonstrate the superiority of NWoS in accuracy, speed, and computational costs.
  arXiv  Detail & Related papers  (2024-06-05T17:59:22Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• A unified scalable framework for causal sweeping strategies for Physics-Informed Neural Networks (PINNs) and their temporal decompositions [22.514769448363754]
  Training challenges in PINNs and XPINNs for time-dependent PDEs are discussed. We propose a new stacked-decomposition method that bridges the gap between PINNs and XPINNs. We also formulate a new time-sweeping collocation point algorithm inspired by the previous PINNs causality.
  arXiv  Detail & Related papers  (2023-02-28T01:19:21Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• Reduced-PINN: An Integration-Based Physics-Informed Neural Networks for Stiff ODEs [0.0]
  Physics-informed neural networks (PINNs) have recently received much attention due to their capabilities in solving both forward and inverse problems. We propose a new PINN architecture, called Reduced-PINN, that utilizes a reduced-order integration method to enable the PINN to solve stiff chemical kinetics.
  arXiv  Detail & Related papers  (2022-08-23T09:20:42Z)
• Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions [68.8204255655161]
  We develop a physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. Localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. We propose criteria for weights in the loss function that produce a more accurate PINN solution than those obtained with the weights selected via other methods.
  arXiv  Detail & Related papers  (2022-08-18T05:00:24Z)
• Neural Basis Functions for Accelerating Solutions to High Mach Euler Equations [63.8376359764052]
  We propose an approach to solving partial differential equations (PDEs) using a set of neural networks. We regress a set of neural networks onto a reduced order Proper Orthogonal Decomposition (POD) basis. These networks are then used in combination with a branch network that ingests the parameters of the prescribed PDE to compute a reduced order approximation to the PDE.
  arXiv  Detail & Related papers  (2022-08-02T18:27:13Z)
• Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs) [1.7403133838762446]
  Physics-Informed Neural Networks (PINNs) are powerful approximators of solutions to nonlinear partial differential equations (PDEs) We propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs) We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.
  arXiv  Detail & Related papers  (2022-04-29T08:33:11Z)
• Monte Carlo PINNs: deep learning approach for forward and inverse problems involving high dimensional fractional partial differential equations [8.378422134042722]
  We introduce a sampling based machine learning approach, Monte Carlo physics informed neural networks (MC-PINNs) for solving forward and inverse fractional partial differential equations (FPDEs) As a generalization of physics informed neural networks (PINNs), our method relies on deep neural network surrogates in addition to an approximation strategy for computing the fractional derivatives of the outputs. We validate the performance of MC-PINNs via several examples that include high dimensional integral fractional Laplacian equations, parametric identification of time-space fractional PDEs, and fractional diffusion equation with random inputs.
  arXiv  Detail & Related papers  (2022-03-16T09:52:05Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Accelerating Physics-Informed Neural Network Training with Prior Dictionaries [7.035456567972667]
  We propose a variant called Prior Dictionary based Physics-Informed Neural Networks (PD-PINNs) PD-PINNs enjoy enhanced representation power on the tasks, which helps to capture features provided by dictionaries. It is proved that under certain mild conditions, the prediction error made by neural networks can be bounded by expected loss of PDEs and boundary conditions.
  arXiv  Detail & Related papers  (2020-04-17T10:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.