Element-wise Multiplication Based Deeper Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2406.04170v4
• Date: Wed, 11 Sep 2024 20:21:36 GMT
• Title: Element-wise Multiplication Based Deeper Physics-Informed Neural Networks
• Authors: Feilong Jiang, Xiaonan Hou, Min Xia,
• Abstract summary: PINNs are a promising framework for resolving partial differential equations (PDEs) Lack of expressive ability and pathology issues are found to prevent the application of PINNs in complex PDEs. We propose Deeper Physics-Informed Neural Network (Deeper-PINN) to resolve these issues.
• Score: 1.8554335256160261
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: As a promising framework for resolving partial differential equations (PDEs), Physics-Informed Neural Networks (PINNs) have received widespread attention from industrial and scientific fields. However, lack of expressive ability and initialization pathology issues are found to prevent the application of PINNs in complex PDEs. In this work, we propose Deeper Physics-Informed Neural Network (Deeper-PINN) to resolve these issues. The element-wise multiplication operation is adopted to transform features into high-dimensional, non-linear spaces. Benefiting from element-wise multiplication operation, Deeper-PINNs can alleviate the initialization pathologies of PINNs and enhance the expressive capability of PINNs. The proposed structure is verified on various benchmarks. The results show that Deeper-PINNs can effectively resolve the initialization pathology and exhibit strong expressive ability.

Related papers

• Multifidelity domain decomposition-based physics-informed neural networks and operators for time-dependent problems [40.46280139210502]
  A combination of multifidelity stacking PINNs and domain decomposition-based finite basis PINNs is employed. It can be observed that the domain decomposition approach clearly improves the PINN and stacking PINN approaches. It is demonstrated that the FBPINN approach can be extended to multifidelity physics-informed deep operator networks.
  arXiv  Detail & Related papers  (2024-01-15T18:32:53Z)
• PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks [22.39904196850583]
  Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions to partial differential equations (PDEs) We introduce a novel Transformer-based framework, termed PINNsFormer, designed to address this limitation. PINNsFormer achieves superior generalization ability and accuracy across various scenarios, including PINNs failure modes and high-dimensional PDEs.
  arXiv  Detail & Related papers  (2023-07-21T18:06:27Z)
• Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach [10.250994619846416]
  We present a new training paradigm referred to as "gradient boosting" (GB) Instead of learning the solution of a given PDE using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This work also unlocks the door to employing ensemble learning techniques in PINNs.
  arXiv  Detail & Related papers  (2023-02-25T19:11:44Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture [77.59766598165551]
  Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation. Here, we propose Auto-PINN, which employs Neural Architecture Search (NAS) techniques to PINN design. A comprehensive set of pre-experiments using standard PDE benchmarks allows us to probe the structure-performance relationship in PINNs.
  arXiv  Detail & Related papers  (2022-05-27T03:24:31Z)
• Revisiting PINNs: Generative Adversarial Physics-informed Neural Networks and Point-weighting Method [70.19159220248805]
  Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs) We propose the generative adversarial neural network (GA-PINN), which integrates the generative adversarial (GA) mechanism with the structure of PINNs. Inspired from the weighting strategy of the Adaboost method, we then introduce a point-weighting (PW) method to improve the training efficiency of PINNs.
  arXiv  Detail & Related papers  (2022-05-18T06:50:44Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next [5.956366179544257]
  Physic-Informed Neural Networks (PINN) are neural networks (NNs) that encode model equations. PINNs are nowadays used to solve PDEs, fractional equations, and integral-differential equations.
  arXiv  Detail & Related papers  (2022-01-14T19:05:44Z)
• Characterizing possible failure modes in physics-informed neural networks [55.83255669840384]
  Recent work in scientific machine learning has developed so-called physics-informed neural network (PINN) models. We demonstrate that, while existing PINN methodologies can learn good models for relatively trivial problems, they can easily fail to learn relevant physical phenomena even for simple PDEs. We show that these possible failure modes are not due to the lack of expressivity in the NN architecture, but that the PINN's setup makes the loss landscape very hard to optimize.
  arXiv  Detail & Related papers  (2021-09-02T16:06:45Z)
• Accelerating Physics-Informed Neural Network Training with Prior Dictionaries [7.035456567972667]
  We propose a variant called Prior Dictionary based Physics-Informed Neural Networks (PD-PINNs) PD-PINNs enjoy enhanced representation power on the tasks, which helps to capture features provided by dictionaries. It is proved that under certain mild conditions, the prediction error made by neural networks can be bounded by expected loss of PDEs and boundary conditions.
  arXiv  Detail & Related papers  (2020-04-17T10:14:41Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.