An Expert's Guide to Training Physics-informed Neural Networks

• URL: http://arxiv.org/abs/2308.08468v1
• Date: Wed, 16 Aug 2023 16:19:25 GMT
• Title: An Expert's Guide to Training Physics-informed Neural Networks
• Authors: Sifan Wang, Shyam Sankaran, Hanwen Wang, Paris Perdikaris
• Abstract summary: Physics-informed neural networks (PINNs) have been popularized as a deep learning framework. PINNs can seamlessly synthesize observational data and partial differential equation (PDE) constraints. We present a series of best practices that can significantly improve the training efficiency and overall accuracy of PINNs.
• Score: 5.198985210238479
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Physics-informed neural networks (PINNs) have been popularized as a deep learning framework that can seamlessly synthesize observational data and partial differential equation (PDE) constraints. Their practical effectiveness however can be hampered by training pathologies, but also oftentimes by poor choices made by users who lack deep learning expertise. In this paper we present a series of best practices that can significantly improve the training efficiency and overall accuracy of PINNs. We also put forth a series of challenging benchmark problems that highlight some of the most prominent difficulties in training PINNs, and present comprehensive and fully reproducible ablation studies that demonstrate how different architecture choices and training strategies affect the test accuracy of the resulting models. We show that the methods and guiding principles put forth in this study lead to state-of-the-art results and provide strong baselines that future studies should use for comparison purposes. To this end, we also release a highly optimized library in JAX that can be used to reproduce all results reported in this paper, enable future research studies, as well as facilitate easy adaptation to new use-case scenarios.

Related papers

• Towards a Foundation Model for Physics-Informed Neural Networks: Multi-PDE Learning with Active Sampling [0.0]
  Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical laws into neural network training. In this work, we explore the potential of a foundation PINN model capable of solving multiple PDEs within a unified architecture.
  arXiv  Detail & Related papers  (2025-02-11T10:12:28Z)
• AL-PINN: Active Learning-Driven Physics-Informed Neural Networks for Efficient Sample Selection in Solving Partial Differential Equations [0.0]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for solving Partial Differential Equations (PDEs) We propose Active Learning-Driven PINNs (AL-PINN), which integrates Uncertainty Quantification (UQ) and Active Learning strategies to optimize sample selection dynamically. Our results demonstrate that AL-PINN achieves comparable or superior accuracy compared to traditional PINNs while reducing the number of required training samples.
  arXiv  Detail & Related papers  (2025-02-06T10:54:28Z)
• Adversarial Learning for Neural PDE Solvers with Sparse Data [4.226449585713182]
  This study introduces a universal learning strategy for neural network PDEs, named Systematic Model Augmentation for Robust Training. By focusing on challenging and improving the model's weaknesses, SMART reduces generalization error during training under data-scarce conditions.
  arXiv  Detail & Related papers  (2024-09-04T04:18:25Z)
• Robust Learning with Progressive Data Expansion Against Spurious Correlation [65.83104529677234]
  We study the learning process of a two-layer nonlinear convolutional neural network in the presence of spurious features. Our analysis suggests that imbalanced data groups and easily learnable spurious features can lead to the dominance of spurious features during the learning process. We propose a new training algorithm called PDE that efficiently enhances the model's robustness for a better worst-group performance.
  arXiv  Detail & Related papers  (2023-06-08T05:44:06Z)
• Adversarial training with informed data selection [53.19381941131439]
  Adrial training is the most efficient solution to defend the network against these malicious attacks. This work proposes a data selection strategy to be applied in the mini-batch training. The simulation results show that a good compromise can be obtained regarding robustness and standard accuracy.
  arXiv  Detail & Related papers  (2023-01-07T12:09:50Z)
• Holistic Deep Learning [3.718942345103135]
  This paper presents a novel holistic deep learning framework that addresses the challenges of vulnerability to input perturbations, overparametrization, and performance instability. The proposed framework holistically improves accuracy, robustness, sparsity, and stability over standard deep learning models.
  arXiv  Detail & Related papers  (2021-10-29T14:46:32Z)
• Deep Active Learning by Leveraging Training Dynamics [57.95155565319465]
  We propose a theory-driven deep active learning method (dynamicAL) which selects samples to maximize training dynamics. We show that dynamicAL not only outperforms other baselines consistently but also scales well on large deep learning models.
  arXiv  Detail & Related papers  (2021-10-16T16:51:05Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms [91.3755431537592]
  We analyze four broad meta-learning strategies which rely on plug-in estimation and pseudo-outcome regression. We highlight how this theoretical reasoning can be used to guide principled algorithm design and translate our analyses into practice.
  arXiv  Detail & Related papers  (2021-01-26T17:11:40Z)
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study [81.11161697133095]
  We take the NER task as a testbed to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models. As a by-product of this paper, we have open-sourced a project that involves a comprehensive summary of recent NER papers.
  arXiv  Detail & Related papers  (2020-01-12T04:33:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.