Neural Networks with Kernel-Weighted Corrective Residuals for Solving
Partial Differential Equations
- URL: http://arxiv.org/abs/2401.03492v1
- Date: Sun, 7 Jan 2024 14:09:42 GMT
- Title: Neural Networks with Kernel-Weighted Corrective Residuals for Solving
Partial Differential Equations
- Authors: Carlos Mora, Amin Yousefpour, Shirin Hosseinmardi, Ramin Bostanabad
- Abstract summary: We introduce kernel-weighted Corrective Residuals (CoRes) to integrate the strengths of kernel methods and deep NNs for solving nonlinear PDE systems.
CoRes consistently outperforms competing methods in solving a broad range of benchmark problems.
We believe our findings have the potential to spark a renewed interest in leveraging kernel methods for solving PDEs.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Physics-informed machine learning (PIML) has emerged as a promising
alternative to conventional numerical methods for solving partial differential
equations (PDEs). PIML models are increasingly built via deep neural networks
(NNs) whose architecture and training process are designed such that the
network satisfies the PDE system. While such PIML models have substantially
advanced over the past few years, their performance is still very sensitive to
the NN's architecture and loss function. Motivated by this limitation, we
introduce kernel-weighted Corrective Residuals (CoRes) to integrate the
strengths of kernel methods and deep NNs for solving nonlinear PDE systems. To
achieve this integration, we design a modular and robust framework which
consistently outperforms competing methods in solving a broad range of
benchmark problems. This performance improvement has a theoretical
justification and is particularly attractive since we simplify the training
process while negligibly increasing the inference costs. Additionally, our
studies on solving multiple PDEs indicate that kernel-weighted CoRes
considerably decrease the sensitivity of NNs to factors such as random
initialization, architecture type, and choice of optimizer. We believe our
findings have the potential to spark a renewed interest in leveraging kernel
methods for solving PDEs.
Related papers
- Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networks [4.216184112447278]
Physics-Informed Neural Networks (PINNs) have emerged as a robust framework for solving Partial Differential Equations (PDEs)
We present a fast implementation of grid-dependent Physics-Informed Kolmogorov-Arnold Networks (PIKANs) for solving PDEs.
We show that these adaptive features significantly enhance training efficiency and solution accuracy.
arXiv Detail & Related papers (2024-07-24T19:55:08Z) - Stable Weight Updating: A Key to Reliable PDE Solutions Using Deep Learning [0.0]
This paper introduces novel residual-based architectures, designed to enhance stability and accuracy in physics-informed neural networks (PINNs)
The architectures augment traditional neural networks by incorporating residual connections, which facilitate smoother weight updates and improve backpropagation efficiency.
The Squared Residual Network, in particular, exhibits robust performance, achieving enhanced stability and accuracy compared to conventional neural networks.
arXiv Detail & Related papers (2024-07-10T05:20:43Z) - RoPINN: Region Optimized Physics-Informed Neural Networks [66.38369833561039]
Physics-informed neural networks (PINNs) have been widely applied to solve partial differential equations (PDEs)
This paper proposes and theoretically studies a new training paradigm as region optimization.
A practical training algorithm, Region Optimized PINN (RoPINN), is seamlessly derived from this new paradigm.
arXiv Detail & Related papers (2024-05-23T09:45:57Z) - Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural
Networks [24.14254861023394]
In this study, we suggest a path that potentially opens up a possibility for physics-informed neural networks (PINNs) to be considered as one such solver.
PINNs have pioneered a proper integration of deep-learning and scientific computing, but they require repetitive time-consuming training of neural networks.
We propose a lightweight low-rank PINNs containing only hundreds of model parameters and an associated hypernetwork-based meta-learning algorithm.
arXiv Detail & Related papers (2023-10-14T08:13:43Z) - Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural
Networks on Coupled Ordinary Differential Equations [64.78260098263489]
In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs)
We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases.
We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
arXiv Detail & Related papers (2022-10-14T15:01:32Z) - Solving Coupled Differential Equation Groups Using PINO-CDE [42.363646159367946]
PINO-CDE is a deep learning framework for solving coupled differential equation groups (CDEs)
Based on the theory of physics-informed neural operator (PINO), PINO-CDE uses a single network for all quantities in a CDEs.
This framework integrates engineering dynamics and deep learning technologies and may reveal a new concept for CDEs solving and uncertainty propagation.
arXiv Detail & Related papers (2022-10-01T08:39:24Z) - PIXEL: Physics-Informed Cell Representations for Fast and Accurate PDE
Solvers [4.1173475271436155]
We propose a new kind of data-driven PDEs solver, physics-informed cell representations (PIXEL)
PIXEL elegantly combines classical numerical methods and learning-based approaches.
We show that PIXEL achieves fast convergence speed and high accuracy.
arXiv Detail & Related papers (2022-07-26T10:46:56Z) - Revisiting PINNs: Generative Adversarial Physics-informed Neural
Networks and Point-weighting Method [70.19159220248805]
Physics-informed neural networks (PINNs) provide a deep learning framework for numerically solving partial differential equations (PDEs)
We propose the generative adversarial neural network (GA-PINN), which integrates the generative adversarial (GA) mechanism with the structure of PINNs.
Inspired from the weighting strategy of the Adaboost method, we then introduce a point-weighting (PW) method to improve the training efficiency of PINNs.
arXiv Detail & Related papers (2022-05-18T06:50:44Z) - Learning Physics-Informed Neural Networks without Stacked
Back-propagation [82.26566759276105]
We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks.
In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation.
Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
arXiv Detail & Related papers (2022-02-18T18:07:54Z) - dNNsolve: an efficient NN-based PDE solver [62.997667081978825]
We introduce dNNsolve, that makes use of dual Neural Networks to solve ODEs/PDEs.
We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions.
arXiv Detail & Related papers (2021-03-15T19:14:41Z) - Large-scale Neural Solvers for Partial Differential Equations [48.7576911714538]
Solving partial differential equations (PDE) is an indispensable part of many branches of science as many processes can be modelled in terms of PDEs.
Recent numerical solvers require manual discretization of the underlying equation as well as sophisticated, tailored code for distributed computing.
We examine the applicability of continuous, mesh-free neural solvers for partial differential equations, physics-informed neural networks (PINNs)
We discuss the accuracy of GatedPINN with respect to analytical solutions -- as well as state-of-the-art numerical solvers, such as spectral solvers.
arXiv Detail & Related papers (2020-09-08T13:26:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.