ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2502.00803v1
• Date: Sun, 02 Feb 2025 13:56:38 GMT
• Title: ProPINN: Demystifying Propagation Failures in Physics-Informed Neural Networks
• Authors: Haixu Wu, Yuezhou Ma, Hang Zhou, Huikun Weng, Jianmin Wang, Mingsheng Long,
• Abstract summary: Physics-informed neural networks (PINNs) have earned high expectations in solving partial differential equations (PDEs) Previous research observed the propagation failure phenomenon of PINNs. This paper provides the first formal and in-depth study of propagation failure and its root cause.
• Score: 71.02216400133858
• License:
• Abstract: Physics-informed neural networks (PINNs) have earned high expectations in solving partial differential equations (PDEs), but their optimization usually faces thorny challenges due to the unique derivative-dependent loss function. By analyzing the loss distribution, previous research observed the propagation failure phenomenon of PINNs, intuitively described as the correct supervision for model outputs cannot ``propagate'' from initial states or boundaries to the interior domain. Going beyond intuitive understanding, this paper provides the first formal and in-depth study of propagation failure and its root cause. Based on a detailed comparison with classical finite element methods, we ascribe the failure to the conventional single-point-processing architecture of PINNs and further prove that propagation failure is essentially caused by the lower gradient correlation of PINN models on nearby collocation points. Compared to superficial loss maps, this new perspective provides a more precise quantitative criterion to identify where and why PINN fails. The theoretical finding also inspires us to present a new PINN architecture, named ProPINN, which can effectively unite the gradient of region points for better propagation. ProPINN can reliably resolve PINN failure modes and significantly surpass advanced Transformer-based models with 46% relative promotion.

Related papers

• General-Kindred Physics-Informed Neural Network to the Solutions of Singularly Perturbed Differential Equations [11.121415128908566]
  We propose the General-Kindred Physics-Informed Neural Network (GKPINN) for solving Singular Perturbation Differential Equations (SPDEs) This approach utilizes prior knowledge of the boundary layer from the equation and establishes a novel network to assist PINN in approxing the boundary layer. The research findings underscore the exceptional performance of our novel approach, GKPINN, which delivers a remarkable enhancement in reducing the $L$ error by two to four orders of magnitude compared to the established PINN methodology.
  arXiv  Detail & Related papers  (2024-08-27T02:03:22Z)
• RoPINN: Region Optimized Physics-Informed Neural Networks [66.38369833561039]
  Physics-informed neural networks (PINNs) have been widely applied to solve partial differential equations (PDEs) This paper proposes and theoretically studies a new training paradigm as region optimization. A practical training algorithm, Region Optimized PINN (RoPINN), is seamlessly derived from this new paradigm.
  arXiv  Detail & Related papers  (2024-05-23T09:45:57Z)
• PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks [22.39904196850583]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions to partial differential equations (PDEs) We introduce a novel Transformer-based framework, termed PINNsFormer, designed to address this limitation. PINNsFormer achieves superior generalization ability and accuracy across various scenarios, including PINNs failure modes and high-dimensional PDEs.
  arXiv  Detail & Related papers  (2023-07-21T18:06:27Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Failure-informed adaptive sampling for PINNs [5.723850818203907]
  Physics-informed neural networks (PINNs) have emerged as an effective technique for solving PDEs in a wide range of domains. Recent research has demonstrated, however, that the performance of PINNs can vary dramatically with different sampling procedures. We present an adaptive approach termed failure-informed PINNs, which is inspired by the viewpoint of reliability analysis.
  arXiv  Detail & Related papers  (2022-10-01T13:34:41Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• PSO-PINN: Physics-Informed Neural Networks Trained with Particle Swarm Optimization [0.0]
  We propose the use of a hybrid particle swarm optimization and gradient descent approach to train PINNs. The resulting PSO-PINN algorithm mitigates the undesired behaviors of PINNs trained with standard gradient descent. Experimental results show that PSO-PINN consistently outperforms a baseline PINN trained with Adam gradient descent.
  arXiv  Detail & Related papers  (2022-02-04T02:21:31Z)
• Robust Learning of Physics Informed Neural Networks [2.86989372262348]
  Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE.
  arXiv  Detail & Related papers  (2021-10-26T00:10:57Z)
• A Theoretical Framework for Target Propagation [75.52598682467817]
  We analyze target propagation (TP), a popular but not yet fully understood alternative to backpropagation (BP) Our theory shows that TP is closely related to Gauss-Newton optimization and thus substantially differs from BP. We provide a first solution to this problem through a novel reconstruction loss that improves feedback weight training.
  arXiv  Detail & Related papers  (2020-06-25T12:07:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.