A Family of Adaptive Activation Functions for Mitigating Failure Modes in Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2603.18328v1
• Date: Wed, 18 Mar 2026 22:24:40 GMT
• Title: A Family of Adaptive Activation Functions for Mitigating Failure Modes in Physics-Informed Neural Networks
• Abstract summary: Motivated by the common failure modes observed in standard PINNs, this work introduces a novel family of adaptive wavelet-based activation functions.<n>The proposed activation functions significantly improve training stability and expressive power by combining trainable wavelet functions with either trainable or fixed hyperbolic tangent and softplus functions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physics-Informed Neural Networks(PINNs) are a powerful and flexible learning framework that has gained significant attention in recent years. It has demonstrated strong performance across a wide range of scientific and engineering problems. In parallel, wavelets have been extensively used as efficient computational tools due to their strong approximation capabilities. Motivated by the common failure modes observed in standard PINNs, this work introduces a novel family of adaptive wavelet-based activation functions. The proposed activation functions significantly improve training stability and expressive power by combining trainable wavelet functions with either trainable or fixed hyperbolic tangent and softplus functions. Five distinct activation functions are developed within the PINN framework and systematically evaluated across four representative classes of partial differential equations (PDEs). Comprehensive comparisons using bar plots demonstrate improved robustness and accuracy compared to traditional activation functions. Furthermore, the proposed approach is validated through direct comparisons with baseline PINNs, transformer-based architectures such as PINNsFormer, and other deep learning models, highlighting its effectiveness and generality.

Related papers

• Feature Interaction Modeling for Physics-Informed Neural Networks and Neural Operators [1.2604738912025477]
  We embed feature interaction modules derived from factorization machines (FMNs) into physics-informed neural networks (PINs) and neural operator learning.<n>We first propose FM-PIN, which explicitly capturestemporal variable interactions and improves approximation accuracy for smooth high-order PDEs.<n>We develop FM-Operator and FM-DeepONet, which are particularly effective for nonlinear conservation laws problems with sharp gradients or discontinuities.
  arXiv  Detail & Related papers  (2026-07-30T18:28:50Z)
• Relay-Assisted Activation-Integrated SIM for Wireless Physical Neural Networks [53.64687377045908]
  Wireless physical neural networks (WPNNs) have emerged as a promising paradigm for performing neural computation directly in the physical layer of wireless systems.<n>We propose a relay-assisted WPNN architecture based on activation-integrated stacked intelligent metasurfaces (AI-SIMs)
  arXiv  Detail & Related papers  (2026-04-05T18:07:13Z)
• Neural Approximation and Its Applications [73.11802084858294]
  We introduce the neural basis function by leveraging an untrained neural network as the basis function.<n>We suggest the neural approximation (NeuApprox) paradigm for multivariate function approximation.<n>Experiments on diverse multi-dimensional images, light field data, videos, traffic data, and point cloud data demonstrate the promising performance of NeuApprox.
  arXiv  Detail & Related papers  (2026-03-04T06:05:43Z)
• Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions [44.44497277876625]
  Physics-Informed Neural Networks (PINNs) are deep learning models that incorporate the governing physical laws of a system into the learning process.<n>We introduce a transfer learning (TL) method to improve the extrapolation capability of PINNs.<n>We demonstrate that our method achieves an average of 40% reduction in relative L2 error and an average of 50% reduction in mean absolute error.
  arXiv  Detail & Related papers  (2025-07-16T22:19:53Z)
• Fourier Multi-Component and Multi-Layer Neural Networks: Unlocking High-Frequency Potential [9.699640804685629]
  We introduce the Fourier Multi-Component and Multi-Layer Neural Network (FMMNN), a novel model that creates a strong synergy between them.<n>We demonstrate that FMMNNs are highly effective and flexible in modeling high-frequency components.<n>We also analyze the optimization landscape of FMMNNs and find it to be much more favorable than that of standard fully connected neural networks.
  arXiv  Detail & Related papers  (2025-02-26T09:12:52Z)
• On the practical applicability of modern DFT functionals for chemical computations. Case study of DM21 applicability for geometry optimization [55.88862563823878]
  This study focuses on evaluating the efficiency of DM21 functional in predicting molecular geometries.<n>We implement geometry optimization in PySCF for the DM21 functional in geometry optimization problem.<n>Our findings reveal both the potential and the current challenges of using neural network functionals for geometry optimization in DFT.
  arXiv  Detail & Related papers  (2025-01-21T14:01:06Z)
• Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equations [0.0]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for solving Partial Differential Equations (PDEs)<n>These limitations impact their accuracy for problems involving rapid oscillations, sharp gradients, and complex boundary behaviors.<n>We investigate learnable activation functions as a solution to these challenges.
  arXiv  Detail & Related papers  (2024-11-22T18:25:13Z)
• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation [49.44309457870649]
  Layer-wise Feedback feedback (LFP) is a novel training principle for neural network-like predictors.<n>LFP decomposes a reward to individual neurons based on their respective contributions.<n>Our method then implements a greedy reinforcing approach helpful parts of the network and weakening harmful ones.
  arXiv  Detail & Related papers  (2023-08-23T10:48:28Z)
• Unification of popular artificial neural network activation functions [0.0]
  We present a unified representation of the most popular neural network activation functions. Adopting Mittag-Leffler functions of fractional calculus, we propose a flexible and compact functional form.
  arXiv  Detail & Related papers  (2023-02-21T21:20:59Z)
• Otimizacao de pesos e funcoes de ativacao de redes neurais aplicadas na previsao de series temporais [0.0]
  We propose the use of a family of free parameter asymmetric activation functions for neural networks. We show that this family of defined activation functions satisfies the requirements of the universal approximation theorem. A methodology for the global optimization of this family of activation functions with free parameter and the weights of the connections between the processing units of the neural network is used.
  arXiv  Detail & Related papers  (2021-07-29T23:32:15Z)
• eQE 2.0: Subsystem DFT Beyond GGA Functionals [58.720142291102135]
  subsystem-DFT (sDFT) can dramatically reduce the computational cost of large-scale electronic structure calculations. The key ingredients of sDFT are the nonadditive kinetic energy and exchange-correlation functionals which dominate it's accuracy. eQE 2.0 delivers excellent interaction energies compared to conventional Kohn-Sham DFT and CCSD(T)
  arXiv  Detail & Related papers  (2021-03-12T22:26:36Z)
• Variational Monte Carlo calculations of $\mathbf{A\leq 4}$ nuclei with an artificial neural-network correlator ansatz [62.997667081978825]
  We introduce a neural-network quantum state ansatz to model the ground-state wave function of light nuclei. We compute the binding energies and point-nucleon densities of $Aleq 4$ nuclei as emerging from a leading-order pionless effective field theory Hamiltonian.
  arXiv  Detail & Related papers  (2020-07-28T14:52:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.