Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)

• URL: http://arxiv.org/abs/2205.01059v2
• Date: Wed, 31 May 2023 02:39:31 GMT
• Title: Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs)
• Authors: Hwijae Son, Sung Woong Cho, Hyung Ju Hwang
• Abstract summary: Physics-Informed Neural Networks (PINNs) are powerful approximators of solutions to nonlinear partial differential equations (PDEs) We propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs) We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.
• Score: 1.7403133838762446
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physics-Informed Neural Networks (PINNs) have become a prominent application of deep learning in scientific computation, as they are powerful approximators of solutions to nonlinear partial differential equations (PDEs). There have been numerous attempts to facilitate the training process of PINNs by adjusting the weight of each component of the loss function, called adaptive loss-balancing algorithms. In this paper, we propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs). We treat the initial and boundary conditions as constraints for the optimization problem of the PDE residual. By employing Augmented Lagrangian relaxation, the constrained optimization problem becomes a sequential max-min problem so that the learnable parameters $\lambda$ adaptively balance each loss component. Our theoretical analysis reveals that the sequence of minimizers of the proposed loss functions converges to an actual solution for the Helmholtz, viscous Burgers, and Klein--Gordon equations. We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.

Related papers

• RoPINN: Region Optimized Physics-Informed Neural Networks [66.38369833561039]
  Physics-informed neural networks (PINNs) have been widely applied to solve partial differential equations (PDEs) This paper proposes and theoretically studies a new training paradigm as region optimization. A practical training algorithm, Region Optimized PINN (RoPINN), is seamlessly derived from this new paradigm.
  arXiv  Detail & Related papers  (2024-05-23T09:45:57Z)
• Solving Forward and Inverse Problems of Contact Mechanics using Physics-Informed Neural Networks [0.0]
  We deploy PINNs in a mixed-variable formulation enhanced by output transformation to enforce hard and soft constraints. We show that PINNs can serve as pure partial equation (PDE) solver, as data-enhanced forward model, and as fast-to-evaluate surrogate model.
  arXiv  Detail & Related papers  (2023-08-24T11:31:24Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• NeuralStagger: Accelerating Physics-constrained Neural PDE Solver with Spatial-temporal Decomposition [67.46012350241969]
  This paper proposes a general acceleration methodology called NeuralStagger. It decomposing the original learning tasks into several coarser-resolution subtasks. We demonstrate the successful application of NeuralStagger on 2D and 3D fluid dynamics simulations.
  arXiv  Detail & Related papers  (2023-02-20T19:36:52Z)
• Replacing Automatic Differentiation by Sobolev Cubatures fastens Physics Informed Neural Nets and strengthens their Approximation Power [0.6091702876917279]
  We present a novel class of approximations for variational losses, being applicable for the training of physics-informed neural nets (PINNs) The loss computation rests on an extension of Gauss-Legendre cubatures, we term Sobolev cubatures, replacing automatic differentiation (A.D.)
  arXiv  Detail & Related papers  (2022-11-23T11:23:08Z)
• Reduced-PINN: An Integration-Based Physics-Informed Neural Networks for Stiff ODEs [0.0]
  Physics-informed neural networks (PINNs) have recently received much attention due to their capabilities in solving both forward and inverse problems. We propose a new PINN architecture, called Reduced-PINN, that utilizes a reduced-order integration method to enable the PINN to solve stiff chemical kinetics.
  arXiv  Detail & Related papers  (2022-08-23T09:20:42Z)
• Physics-Informed Neural Network Method for Parabolic Differential Equations with Sharply Perturbed Initial Conditions [68.8204255655161]
  We develop a physics-informed neural network (PINN) model for parabolic problems with a sharply perturbed initial condition. Localized large gradients in the ADE solution make the (common in PINN) Latin hypercube sampling of the equation's residual highly inefficient. We propose criteria for weights in the loss function that produce a more accurate PINN solution than those obtained with the weights selected via other methods.
  arXiv  Detail & Related papers  (2022-08-18T05:00:24Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Physics-Informed Neural Operator for Learning Partial Differential Equations [55.406540167010014]
  PINO is the first hybrid approach incorporating data and PDE constraints at different resolutions to learn the operator. The resulting PINO model can accurately approximate the ground-truth solution operator for many popular PDE families.
  arXiv  Detail & Related papers  (2021-11-06T03:41:34Z)
• Multi-Objective Loss Balancing for Physics-Informed Deep Learning [0.0]
  We observe the role of correctly weighting the combination of multiple competitive loss functions for training PINNs effectively. We propose a novel self-adaptive loss balancing of PINNs called ReLoBRaLo. Our simulation studies show that ReLoBRaLo training is much faster and achieves higher accuracy than training PINNs with other balancing methods.
  arXiv  Detail & Related papers  (2021-10-19T09:00:12Z)
• Physics and Equality Constrained Artificial Neural Networks: Application to Partial Differential Equations [1.370633147306388]
  Physics-informed neural networks (PINNs) have been proposed to learn the solution of partial differential equations (PDE) Here, we show that this specific way of formulating the objective function is the source of severe limitations in the PINN approach. We propose a versatile framework that can tackle both inverse and forward problems.
  arXiv  Detail & Related papers  (2021-09-30T05:55:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.