Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise

• URL: http://arxiv.org/abs/2408.09340v2
• Date: Sun, 04 May 2025 20:25:51 GMT
• Title: Enhanced BPINN Training Convergence in Solving General and Multi-scale Elliptic PDEs with Noise
• Authors: Yilong Hou, Xi'an Li, Jinran Wu, You-Gan Wang,
• Abstract summary: We develop a robust multi-scale BPINN (dubbed MBPINN) method by integrating multi-scale deep neural networks (MscaleDNN) and the BPINN framework.<n>We demonstrate the applicability and performance of the proposed method through some general Poisson and multi-scale elliptic problems in one and two-dimensional Euclidean spaces.
• Score: 32.49886401812083
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian Physics Informed Neural Networks (BPINN) have attracted considerable attention for inferring the system states and physical parameters of differential equations according to noisy observations. However, in practice, Hamiltonian Monte Carlo (HMC) used to estimate the internal parameters of the solver for BPINN often encounters these troubles including poor performance and awful convergence for a given step size used to adjust the momentum of those parameters. To address the convergence of HMC for the BPINN method and extend its application scope to multi-scale partial differential equations (PDE), we develop a robust multi-scale BPINN (dubbed MBPINN) method by integrating multi-scale deep neural networks (MscaleDNN) and the BPINN framework. In this newly proposed MBPINN method, we reframe HMC with Stochastic Gradient Descent (SGD) to ensure the most ``likely'' estimation is always provided, and we configure its solver as a Fourier feature mapping-induced MscaleDNN. This novel method offers several key advantages: (1) it is more robust than HMC, (2) it incurs less computational cost than HMC, and (3) it is more flexible for complex problems. We demonstrate the applicability and performance of the proposed method through some general Poisson and multi-scale elliptic problems in one and two-dimensional Euclidean spaces. Our findings indicate that the proposed method can avoid HMC failures and provide valid results. Additionally, our method is capable of handling complex elliptic PDE and producing comparable results for general elliptic PDE under the case of lower signal-to-noise rate. These findings suggest that our proposed approach has great potential for physics-informed machine learning for parameter estimation and solution recovery in the case of ill-posed problems.

Related papers

• BiLO: Bilevel Local Operator Learning for PDE Inverse Problems. Part II: Efficient Uncertainty Quantification with Low-Rank Adaptation [9.229577043169224]
  Uncertainty quantification and inverse problems governed by partial differential equations (PDEs) are central to a wide range of scientific and engineering applications.<n>We extend Bilevel Local Operator Learning (BiLO) for PDE-constrained optimization problems developed in Part 1 to the Bayesian inference framework.
  arXiv  Detail & Related papers  (2025-07-22T21:20:20Z)
• Enhancing Physics-Informed Neural Networks with a Hybrid Parallel Kolmogorov-Arnold and MLP Architecture [0.0]
  We propose a novel architecture that integrates parallelized KAN and branches within a unified PINN framework. The HPKM-PINN introduces a scaling factor xi, to optimally balance the complementary strengths of KAN's interpretable function approximation and numerical's nonlinear learning. These findings highlight the HPKM-PINN's ability to leverage KAN's interpretability and robustness, positioning it as a versatile and scalable tool for solving complex PDE-driven problems.
  arXiv  Detail & Related papers  (2025-03-30T02:59:32Z)
• Solving Partial Differential Equations with Random Feature Models [1.3597551064547502]
  We introduce a random feature based framework toward efficiently solving PDEs.<n>In contrast to the state-of-the-art solvers that face challenges with a large number of collocation points, our proposed method reduces the computational complexity.
  arXiv  Detail & Related papers  (2024-12-31T05:48:31Z)
• A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning [74.80956524812714]
  We tackle the general differentiable meta learning problem that is ubiquitous in modern deep learning. These problems are often formalized as Bi-Level optimizations (BLO) We introduce a novel perspective by turning a given BLO problem into a ii optimization, where the inner loss function becomes a smooth distribution, and the outer loss becomes an expected loss over the inner distribution.
  arXiv  Detail & Related papers  (2024-10-14T12:10:06Z)
• Randomized Physics-Informed Neural Networks for Bayesian Data Assimilation [44.99833362998488]
  We propose a randomized physics-informed neural network (PINN) or rPINN method for uncertainty quantification in inverse partial differential equation (PDE) problems with noisy data. For the linear Poisson equation, HMC and rPINN produce similar distributions, but rPINN is on average 27 times faster than HMC. For the non-linear Poison and diffusion equations, the HMC method fails to converge because a single HMC chain cannot sample multiple modes of the posterior distribution of the PINN parameters in a reasonable amount of time.
  arXiv  Detail & Related papers  (2024-07-05T16:16:47Z)
• Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods [52.0617030129699]
  We introduce a novel theoretical framework for analyzing the effectiveness of DeepMatching Networks and Reinforcement Learning methods. Our main contribution holds for a broad class of problems including Max-and Min-Cut, Max-$k$-Bipartite-Bi, Maximum-Weight-Bipartite-Bi, and Traveling Salesman Problem. As a byproduct of our analysis we introduce a novel regularization process over vanilla descent and provide theoretical and experimental evidence that it helps address vanishing-gradient issues and escape bad stationary points.
  arXiv  Detail & Related papers  (2023-10-08T23:39:38Z)
• Efficient Bayesian Physics Informed Neural Networks for Inverse Problems via Ensemble Kalman Inversion [0.0]
  We present a new efficient inference algorithm for B-PINNs that uses Ensemble Kalman Inversion (EKI) for high-dimensional inference tasks. We find that our proposed method can achieve inference results with informative uncertainty estimates comparable to Hamiltonian Monte Carlo (HMC)-based B-PINNs with a much reduced computational cost.
  arXiv  Detail & Related papers  (2023-03-13T18:15:26Z)
• Implicit Stochastic Gradient Descent for Training Physics-informed Neural Networks [51.92362217307946]
  Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems. PINNs are trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit gradient descent (ISGD) method to train PINNs for improving the stability of training process.
  arXiv  Detail & Related papers  (2023-03-03T08:17:47Z)
• Monte Carlo Neural PDE Solver for Learning PDEs via Probabilistic Representation [59.45669299295436]
  We propose a Monte Carlo PDE solver for training unsupervised neural solvers. We use the PDEs' probabilistic representation, which regards macroscopic phenomena as ensembles of random particles. Our experiments on convection-diffusion, Allen-Cahn, and Navier-Stokes equations demonstrate significant improvements in accuracy and efficiency.
  arXiv  Detail & Related papers  (2023-02-10T08:05:19Z)
• Bayesian deep learning framework for uncertainty quantification in high dimensions [6.282068591820945]
  We develop a novel deep learning method for uncertainty quantification in partial differential equations based on Bayesian neural network (BNN) and Hamiltonian Monte Carlo (HMC) A BNN efficiently learns the posterior distribution of the parameters in deep neural networks by performing Bayesian inference on the network parameters. The posterior distribution is efficiently sampled using HMC to quantify uncertainties in the system.
  arXiv  Detail & Related papers  (2022-10-21T05:20:06Z)
• MRF-PINN: A Multi-Receptive-Field convolutional physics-informed neural network for solving partial differential equations [6.285167805465505]
  Physics-informed neural networks (PINN) can achieve lower development and solving cost than traditional partial differential equation (PDE) solvers. Due to the advantages of parameter sharing, spatial feature extraction and low inference cost, convolutional neural networks (CNN) are increasingly used in PINN.
  arXiv  Detail & Related papers  (2022-09-06T12:26:22Z)
• Monte Carlo PINNs: deep learning approach for forward and inverse problems involving high dimensional fractional partial differential equations [8.378422134042722]
  We introduce a sampling based machine learning approach, Monte Carlo physics informed neural networks (MC-PINNs) for solving forward and inverse fractional partial differential equations (FPDEs) As a generalization of physics informed neural networks (PINNs), our method relies on deep neural network surrogates in addition to an approximation strategy for computing the fractional derivatives of the outputs. We validate the performance of MC-PINNs via several examples that include high dimensional integral fractional Laplacian equations, parametric identification of time-space fractional PDEs, and fractional diffusion equation with random inputs.
  arXiv  Detail & Related papers  (2022-03-16T09:52:05Z)
• Learning Physics-Informed Neural Networks without Stacked Back-propagation [82.26566759276105]
  We develop a novel approach that can significantly accelerate the training of Physics-Informed Neural Networks. In particular, we parameterize the PDE solution by the Gaussian smoothed model and show that, derived from Stein's Identity, the second-order derivatives can be efficiently calculated without back-propagation. Experimental results show that our proposed method can achieve competitive error compared to standard PINN training but is two orders of magnitude faster.
  arXiv  Detail & Related papers  (2022-02-18T18:07:54Z)
• dNNsolve: an efficient NN-based PDE solver [62.997667081978825]
  We introduce dNNsolve, that makes use of dual Neural Networks to solve ODEs/PDEs. We show that dNNsolve is capable of solving a broad range of ODEs/PDEs in 1, 2 and 3 spacetime dimensions.
  arXiv  Detail & Related papers  (2021-03-15T19:14:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.