RBF-PINN: Non-Fourier Positional Embedding in Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2402.08367v2
• Date: Mon, 29 Apr 2024 10:38:19 GMT
• Title: RBF-PINN: Non-Fourier Positional Embedding in Physics-Informed Neural Networks
• Authors: Chengxi Zeng, Tilo Burghardt, Alberto M Gambaruto,
• Abstract summary: We highlight the limitations of widely used Fourier-based feature mapping in certain situations. We suggest the use of the conditionally positive definite Radial Basis Function. Our method can be seamlessly integrated into coordinate-based input neural networks.
• Score: 1.9819034119774483
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: While many recent Physics-Informed Neural Networks (PINNs) variants have had considerable success in solving Partial Differential Equations, the empirical benefits of feature mapping drawn from the broader Neural Representations research have been largely overlooked. We highlight the limitations of widely used Fourier-based feature mapping in certain situations and suggest the use of the conditionally positive definite Radial Basis Function. The empirical findings demonstrate the effectiveness of our approach across a variety of forward and inverse problem cases. Our method can be seamlessly integrated into coordinate-based input neural networks and contribute to the wider field of PINNs research.

Related papers

• PINNfluence: Influence Functions for Physics-Informed Neural Networks [47.27512105490682]
  Physics-informed neural networks (PINNs) have emerged as a flexible and promising application of deep learning to partial differential equations in the physical sciences. We explore the application of influence functions (IFs) to validate and debug PINNs post-hoc.
  arXiv  Detail & Related papers  (2024-09-13T16:23:17Z)
• Element-wise Multiplication Based Deeper Physics-Informed Neural Networks [1.8554335256160261]
  PINNs are a promising framework for resolving partial differential equations (PDEs) Lack of expressive ability and pathology issues are found to prevent the application of PINNs in complex PDEs. We propose Deeper Physics-Informed Neural Network (Deeper-PINN) to resolve these issues.
  arXiv  Detail & Related papers  (2024-06-06T15:27:52Z)
• Feature Mapping in Physics-Informed Neural Networks (PINNs) [1.9819034119774483]
  We study the training dynamics of PINNs with a feature mapping layer via the limiting Conjugate Kernel and Neural Tangent Kernel. We propose conditionally positive definite Radial Basis Function as a better alternative.
  arXiv  Detail & Related papers  (2024-02-10T13:51:09Z)
• Multifidelity domain decomposition-based physics-informed neural networks and operators for time-dependent problems [40.46280139210502]
  A combination of multifidelity stacking PINNs and domain decomposition-based finite basis PINNs is employed. It can be observed that the domain decomposition approach clearly improves the PINN and stacking PINN approaches. It is demonstrated that the FBPINN approach can be extended to multifidelity physics-informed deep operator networks.
  arXiv  Detail & Related papers  (2024-01-15T18:32:53Z)
• Spherical Inducing Features for Orthogonally-Decoupled Gaussian Processes [7.4468224549568705]
  Gaussian processes (GPs) are often compared unfavorably to deep neural networks (NNs) for lacking the ability to learn representations. Recent efforts to bridge the gap between GPs and deep NNs have yielded a new class of inter-domain variational GPs in which the inducing variables correspond to hidden units of a feedforward NN.
  arXiv  Detail & Related papers  (2023-04-27T09:00:02Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks [72.09574528342732]
  Physics-informed neural network (PINN) algorithms have shown promising results in solving a wide range of problems involving partial differential equations (PDEs) They often fail to converge to desirable solutions when the target function contains high-frequency features, due to a phenomenon known as spectral bias. In the present work, we exploit neural tangent kernels (NTKs) to investigate the training dynamics of PINNs evolving under gradient descent with momentum (SGDM)
  arXiv  Detail & Related papers  (2022-06-29T19:03:10Z)
• The Spectral Bias of Polynomial Neural Networks [63.27903166253743]
  Polynomial neural networks (PNNs) have been shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Previous studies have revealed that neural networks demonstrate a $textitspectral bias$ towards low-frequency functions, which yields faster learning of low-frequency components during training. Inspired by such studies, we conduct a spectral analysis of the Tangent Kernel (NTK) of PNNs. We find that the $Pi$-Net family, i.e., a recently proposed parametrization of PNNs, speeds up the
  arXiv  Detail & Related papers  (2022-02-27T23:12:43Z)
• Physics informed neural networks for continuum micromechanics [68.8204255655161]
  Recently, physics informed neural networks have successfully been applied to a broad variety of problems in applied mathematics and engineering. Due to the global approximation, physics informed neural networks have difficulties in displaying localized effects and strong non-linear solutions by optimization. It is shown, that the domain decomposition approach is able to accurately resolve nonlinear stress, displacement and energy fields in heterogeneous microstructures obtained from real-world $mu$CT-scans.
  arXiv  Detail & Related papers  (2021-10-14T14:05:19Z)
• On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks [0.0]
  We show that neural networks (PINNs) struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. We construct novel architectures that employ multi-scale random observational features and justify how such coordinate embedding layers can lead to robust and accurate PINN models.
  arXiv  Detail & Related papers  (2020-12-18T04:19:30Z)
• Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks [107.77595511218429]
  In this paper, we investigate the empirical Rademacher complexity related to intermediate layers of deep neural networks. We propose a feature distortion method (Disout) for addressing the aforementioned problem. The superiority of the proposed feature map distortion for producing deep neural network with higher testing performance is analyzed and demonstrated.
  arXiv  Detail & Related papers  (2020-02-23T13:59:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.