Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks

• URL: http://arxiv.org/abs/2510.27018v1
• Date: Thu, 30 Oct 2025 21:45:10 GMT
• Title: Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks
• Authors: Alexander Heinlein, Taniya Kapoor,
• Abstract summary: Approxing boundary value problems governed by partial differential equations with neural networks is challenging, largely due to the difficult training process.<n>This difficulty can be partly explained by the spectral bias, that is, the slower convergence of high-frequency components.<n>We combine this localization with the Gauss-Newton method as the gradient to obtain faster convergence than gradient-based schemes such as Adam.<n>Our numerical results indicate that combining localization and Gauss-Newton optimization is promising for neural network-based solvers for partial differential equations.
• Score: 47.614449195824335
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Approximating the solutions of boundary value problems governed by partial differential equations with neural networks is challenging, largely due to the difficult training process. This difficulty can be partly explained by the spectral bias, that is, the slower convergence of high-frequency components, and can be mitigated by localizing neural networks via (overlapping) domain decomposition. We combine this localization with the Gauss-Newton method as the optimizer to obtain faster convergence than gradient-based schemes such as Adam; this comes at the cost of solving an ill-conditioned linear system in each iteration. Domain decomposition induces a block-sparse structure in the otherwise dense Gauss-Newton system, reducing the computational cost per iteration. Our numerical results indicate that combining localization and Gauss-Newton optimization is promising for neural network-based solvers for partial differential equations.

Related papers

• DInf-Grid: A Neural Differential Equation Solver with Differentiable Feature Grids [73.28614344779076]
  We present a differentiable grid-based representation for efficiently solving differential equations (DEs)<n>Our results demonstrate a 5-20x speed-up over coordinate-based methods, solving differential equations in seconds or minutes while maintaining comparable accuracy and compactness.
  arXiv  Detail & Related papers  (2026-01-15T18:59:57Z)
• Gradient Descent as a Perceptron Algorithm: Understanding Dynamics and Implicit Acceleration [67.12978375116599]
  We show that the steps of gradient descent (GD) reduce to those of generalized perceptron algorithms.<n>This helps explain the optimization dynamics and the implicit acceleration phenomenon observed in neural networks.
  arXiv  Detail & Related papers  (2025-12-12T14:16:35Z)
• Chebyshev Spectral Neural Networks for Solving Partial Differential Equations [0.0]
  The study uses a feedforward neural network model and error backpropagation principles, utilizing automatic differentiation (AD) to compute the loss function. The numerical efficiency and accuracy of the CSNN model are investigated through testing on elliptic partial differential equations, and it is compared with the well-known Physics-Informed Neural Network(PINN) method.
  arXiv  Detail & Related papers  (2024-06-06T05:31:45Z)
• Stochastic Gradient Descent for Gaussian Processes Done Right [86.83678041846971]
  We show that when emphdone right -- by which we mean using specific insights from optimisation and kernel communities -- gradient descent is highly effective. We introduce a emphstochastic dual descent algorithm, explain its design in an intuitive manner and illustrate the design choices. Our method places Gaussian process regression on par with state-of-the-art graph neural networks for molecular binding affinity prediction.
  arXiv  Detail & Related papers  (2023-10-31T16:15:13Z)
• Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks [4.604003661048267]
  Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. We propose a comprehensive theoretical framework that sheds light on this important problem. We derive an integro-differential equation that governs PINN prediction in the large data-set limit.
  arXiv  Detail & Related papers  (2023-07-12T18:00:02Z)
• A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms [64.3064050603721]
  We generalize Runge-Kutta neural network to a recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields similar iterations to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta solvers for ordinary differential equations.
  arXiv  Detail & Related papers  (2022-11-22T16:30:33Z)
• Neuro-symbolic partial differential equation solver [0.0]
  We present a strategy for developing mesh-free neuro-symbolic partial differential equation solvers from numerical discretizations found in scientific computing. This strategy is unique in that it can be used to efficiently train neural network surrogate models for the solution functions and the differential operators.
  arXiv  Detail & Related papers  (2022-10-25T22:56:43Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• Hierarchical Learning to Solve Partial Differential Equations Using Physics-Informed Neural Networks [2.0305676256390934]
  We propose a hierarchical approach to improve the convergence rate and accuracy of the neural network solution to partial differential equations. We validate the efficiency and robustness of the proposed hierarchical approach through a suite of linear and nonlinear partial differential equations.
  arXiv  Detail & Related papers  (2021-12-02T13:53:42Z)
• Deep neural network for solving differential equations motivated by Legendre-Galerkin approximation [16.64525769134209]
  We explore the performance and accuracy of various neural architectures on both linear and nonlinear differential equations. We implement a novel Legendre-Galerkin Deep Neural Network (LGNet) algorithm to predict solutions to differential equations.
  arXiv  Detail & Related papers  (2020-10-24T20:25:09Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.