Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions

• URL: http://arxiv.org/abs/2507.12659v1
• Date: Wed, 16 Jul 2025 22:19:53 GMT
• Title: Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions
• Authors: Athanasios Papastathopoulos-Katsaros, Alexandra Stavrianidi, Zhandong Liu,
• Abstract summary: Physics-Informed Neural Networks (PINNs) are deep learning models that incorporate the governing physical laws of a system into the learning process.<n>We introduce a transfer learning (TL) method to improve the extrapolation capability of PINNs.<n>We demonstrate that our method achieves an average of 40% reduction in relative L2 error and an average of 50% reduction in mean absolute error.
• Score: 44.44497277876625
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physics-Informed Neural Networks (PINNs) are deep learning models that incorporate the governing physical laws of a system into the learning process, making them well-suited for solving complex scientific and engineering problems. Recently, PINNs have gained widespread attention as a powerful framework for combining physical principles with data-driven modeling to improve prediction accuracy. Despite their successes, however, PINNs often exhibit poor extrapolation performance outside the training domain and are highly sensitive to the choice of activation functions (AFs). In this paper, we introduce a transfer learning (TL) method to improve the extrapolation capability of PINNs. Our approach applies transfer learning (TL) within an extended training domain, using only a small number of carefully selected collocation points. Additionally, we propose an adaptive AF that takes the form of a linear combination of standard AFs, which improves both the robustness and accuracy of the model. Through a series of experiments, we demonstrate that our method achieves an average of 40% reduction in relative L2 error and an average of 50% reduction in mean absolute error in the extrapolation domain, all without a significant increase in computational cost. The code is available at https://github.com/LiuzLab/PINN-extrapolation .

Related papers

• AL-PINN: Active Learning-Driven Physics-Informed Neural Networks for Efficient Sample Selection in Solving Partial Differential Equations [0.0]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for solving Partial Differential Equations (PDEs)<n>We propose Active Learning-Driven PINNs (AL-PINN), which integrates Uncertainty Quantification (UQ) and Active Learning strategies to optimize sample selection dynamically.<n>Our results demonstrate that AL-PINN achieves comparable or superior accuracy compared to traditional PINNs while reducing the number of required training samples.
  arXiv  Detail & Related papers  (2025-02-06T10:54:28Z)
• Domain-decoupled Physics-informed Neural Networks with Closed-form Gradients for Fast Model Learning of Dynamical Systems [2.8730926763860687]
  Physics-informed neural networks (PINNs) are trained using physical equations and can incorporate unmodeled effects by learning from data. We introduce the domain-decoupled physics-informed neural network (DD-PINN) to address current limitations of PINC in handling large and complex nonlinear dynamical systems.
  arXiv  Detail & Related papers  (2024-08-27T10:54:51Z)
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation [49.44309457870649]
  Layer-wise Feedback feedback (LFP) is a novel training principle for neural network-like predictors.<n>LFP decomposes a reward to individual neurons based on their respective contributions.<n>Our method then implements a greedy reinforcing approach helpful parts of the network and weakening harmful ones.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Transfer Learning with Physics-Informed Neural Networks for Efficient Simulation of Branched Flows [1.1470070927586016]
  Physics-Informed Neural Networks (PINNs) offer a promising approach to solving differential equations. We adopt a recently developed transfer learning approach for PINNs and introduce a multi-head model. We show that our methods provide significant computational speedups in comparison to standard PINNs trained from scratch.
  arXiv  Detail & Related papers  (2022-11-01T01:50:00Z)
• Adaptive Self-supervision Algorithms for Physics-informed Neural Networks [59.822151945132525]
  Physics-informed neural networks (PINNs) incorporate physical knowledge from the problem domain as a soft constraint on the loss function. We study the impact of the location of the collocation points on the trainability of these models. We propose a novel adaptive collocation scheme which progressively allocates more collocation points to areas where the model is making higher errors.
  arXiv  Detail & Related papers  (2022-07-08T18:17:06Z)
• Certified machine learning: A posteriori error estimation for physics-informed neural networks [0.0]
  PINNs are known to be robust for smaller training sets, derive better generalization problems, and are faster to train. We show that using PINNs in comparison with purely data-driven neural networks is not only favorable for training performance but allows us to extract significant information on the quality of the approximated solution.
  arXiv  Detail & Related papers  (2022-03-31T14:23:04Z)
• A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations [69.73803123972297]
  We propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems.
  arXiv  Detail & Related papers  (2020-12-21T17:39:51Z)
• Optimization-driven Machine Learning for Intelligent Reflecting Surfaces Assisted Wireless Networks [82.33619654835348]
  Intelligent surface (IRS) has been employed to reshape the wireless channels by controlling individual scattering elements' phase shifts. Due to the large size of scattering elements, the passive beamforming is typically challenged by the high computational complexity. In this article, we focus on machine learning (ML) approaches for performance in IRS-assisted wireless networks.
  arXiv  Detail & Related papers  (2020-08-29T08:39:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.