Collocation-based Robust Variational Physics-Informed Neural Networks (CRVPINN)

• URL: http://arxiv.org/abs/2401.02300v3
• Date: Wed, 16 Oct 2024 09:20:47 GMT
• Title: Collocation-based Robust Variational Physics-Informed Neural Networks (CRVPINN)
• Authors: Marcin Łoś, Tomasz Służalec, Paweł Maczuga, Askold Vilkha, Carlos Uriarte, Maciej Paszyński,
• Abstract summary: Physics-Informed Neural Networks (PINNs) have been successfully applied to solve Partial Differential Equations (PDEs) Recent work of Robust Variational Physics-Informed Neural Networks (RVPINNs) highlights the importance of conveniently translating the norms of the underlying continuum-level spaces to the discrete level. In this work, we accelerate the implementation of RVPINN, establishing a LU factorization of sparse Gram matrix in a kind of point-collocation scheme with the same spirit as original PINNs.
• Score: 0.0
• License:
• Abstract: Physics-Informed Neural Networks (PINNs) have been successfully applied to solve Partial Differential Equations (PDEs). Their loss function is founded on a strong residual minimization scheme. Variational Physics-Informed Neural Networks (VPINNs) are their natural extension to weak variational settings. In this context, the recent work of Robust Variational Physics-Informed Neural Networks (RVPINNs) highlights the importance of conveniently translating the norms of the underlying continuum-level spaces to the discrete level. Otherwise, VPINNs might become unrobust, implying that residual minimization might be highly uncorrelated with a desired minimization of the error in the energy norm. However, applying this robustness to VPINNs typically entails dealing with the inverse of a Gram matrix, usually producing slow convergence speeds during training. In this work, we accelerate the implementation of RVPINN, establishing a LU factorization of sparse Gram matrix in a kind of point-collocation scheme with the same spirit as original PINNs. We call out method the Collocation-based Robust Variational Physics Informed Neural Networks (CRVPINN). We test our efficient CRVPINN algorithm on Laplace, advection-diffusion, and Stokes problems in two spatial dimensions.

Related papers

• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• Replacing Automatic Differentiation by Sobolev Cubatures fastens Physics Informed Neural Nets and strengthens their Approximation Power [0.6091702876917279]
  We present a novel class of approximations for variational losses, being applicable for the training of physics-informed neural nets (PINNs) The loss computation rests on an extension of Gauss-Legendre cubatures, we term Sobolev cubatures, replacing automatic differentiation (A.D.)
  arXiv  Detail & Related papers  (2022-11-23T11:23:08Z)
• Reduced-PINN: An Integration-Based Physics-Informed Neural Networks for Stiff ODEs [0.0]
  Physics-informed neural networks (PINNs) have recently received much attention due to their capabilities in solving both forward and inverse problems. We propose a new PINN architecture, called Reduced-PINN, that utilizes a reduced-order integration method to enable the PINN to solve stiff chemical kinetics.
  arXiv  Detail & Related papers  (2022-08-23T09:20:42Z)
• Adaptive Self-supervision Algorithms for Physics-informed Neural Networks [59.822151945132525]
  Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function. We study the impact of the location of the collocation points on the trainability of these models. We propose a novel adaptive collocation scheme which progressively allocates more collocation points to areas where the model is making higher errors.
  arXiv  Detail & Related papers  (2022-07-08T18:17:06Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs) [1.7403133838762446]
  Physics-Informed Neural Networks (PINNs) are powerful approximators of solutions to nonlinear partial differential equations (PDEs) We propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs) We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.
  arXiv  Detail & Related papers  (2022-04-29T08:33:11Z)
• Learning in Sinusoidal Spaces with Physics-Informed Neural Networks [22.47355575565345]
  A physics-informed neural network (PINN) uses physics-augmented loss functions to ensure its output is consistent with fundamental physics laws. It turns out to be difficult to train an accurate PINN model for many problems in practice.
  arXiv  Detail & Related papers  (2021-09-20T07:42:41Z)
• Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism [1.6114012813668932]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising application of deep neural networks to the numerical solution of nonlinear partial differential equations (PDEs) We propose a fundamentally new way to train PINNs adaptively, where the adaptation weights are fully trainable and applied to each training point individually. In numerical experiments with several linear and nonlinear benchmark problems, the SA-PINN outperformed other state-of-the-art PINN algorithm in L2 error.
  arXiv  Detail & Related papers  (2020-09-07T04:07:52Z)
• Improve Generalization and Robustness of Neural Networks via Weight Scale Shifting Invariant Regularizations [52.493315075385325]
  We show that a family of regularizers, including weight decay, is ineffective at penalizing the intrinsic norms of weights for networks with homogeneous activation functions. We propose an improved regularizer that is invariant to weight scale shifting and thus effectively constrains the intrinsic norm of a neural network.
  arXiv  Detail & Related papers  (2020-08-07T02:55:28Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.