Variational Boosting for Physics-Informed Neural Networks

• URL: http://arxiv.org/abs/2607.23940v1
• Date: Mon, 27 Jul 2026 02:32:22 GMT
• Title: Variational Boosting for Physics-Informed Neural Networks
• Abstract summary: monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability.<n>We introduce a variational boosting framework in which solutions are constructed additively in function space.<n>This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent.
• Score: 0.794957965474334
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-Informed Neural Networks (PINNs) solve differential equations by minimizing the residual of a nonlinear operator over a neural parameterization of the solution. However, monolithic PINNs often suffer from ill-conditioning, spectral bias, and optimization instability. We introduce a variational boosting framework in which solutions are constructed additively in function space. Each stage trains a weak learner whose converged correction satisfies a local orthogonality condition, equivalent to a projected functional gradient descent step onto the tangent space of the network's function manifold. Because each correction network is deliberately small, the restricted minimization admits full Newton or conjugate gradient updates, which are typically infeasible in large PINNs. The resulting method separates global nonlinear refinement into a sequence of well-conditioned subproblems while preserving the full variational structure of the operator. This framework provides a geometric interpretation of multi-stage PINNs as projected functional gradient descent and enables stable second-order optimization for nonlinear differential equations.

Related papers

• Linearized PINN with pretrained nonlinear layers [48.01541150431857]
  lPINN is a reduced-order neural basis method for forward and inverse differential equations.<n>We evaluate lPINN on forward and inverse problems for the advection-diffusion equation, Burgers' equation, and the nonlinear pendulum equation.
  arXiv  Detail & Related papers  (2026-09-14T02:18:00Z)
• Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training for Nonlinear Differential Equations [7.927063711454594]
  Physics-In Networks (SCNs) have emerged as a transformative paradigm for solving complex differential equations.<n>Our work provides a highly efficient and scalable foundation for next-generation, real-time Scientific Machine applications.
  arXiv  Detail & Related papers  (2026-08-27T02:40:11Z)
• A Physics-Informed Neural Network with a Modified Lorentzian Activation for Nonlocal Gradient-Flow Equations in Dynamic Density Functional Theory [0.8602553195689513]
  We develop a physics-informed neural network (PINN) framework for nonlocal partial differential equations.<n>These equations involve nonlinearities, nonlocal interaction terms, and an underlying gradient-flow structure.<n>We adapt the PINN methodology to DDFT gradient-flow equations and introduce two computational components.
  arXiv  Detail & Related papers  (2026-06-24T22:01:13Z)
• Randomized Neural Networks for Integro-Differential Equations with Application to Neutron Transport [10.653981193573676]
  In work, we present randomized neural networks (RaNNs) as a simulation of integrodifferential equations.<n>Because the RaNN is intrinsically dense globally supported random approximation, the nonlocal integral operator does not introduce an additional loss of sparsity.
  arXiv  Detail & Related papers  (2026-04-15T13:05:28Z)
• Neural-POD: A Plug-and-Play Neural Operator Framework for Infinite-Dimensional Functional Nonlinear Proper Orthogonal Decomposition [7.1950116347185995]
  We propose the Neural Proper Orthogonal Decomposition (Neural-POD), a plug-and-play neural operator framework.<n>Neural-POD formulates basis construction as a sequence of residual minimization problems solved through neural network training.<n>We demonstrate the robustness of Neural-POD with different complex resolutions, including the Burgers' and NavierStokes equations.
  arXiv  Detail & Related papers  (2026-02-17T15:01:40Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Enhanced Physics-Informed Neural Networks with Augmented Lagrangian Relaxation Method (AL-PINNs) [1.7403133838762446]
  Physics-Informed Neural Networks (PINNs) are powerful approximators of solutions to nonlinear partial differential equations (PDEs) We propose an Augmented Lagrangian relaxation method for PINNs (AL-PINNs) We demonstrate through various numerical experiments that AL-PINNs yield a much smaller relative error compared with that of state-of-the-art adaptive loss-balancing algorithms.
  arXiv  Detail & Related papers  (2022-04-29T08:33:11Z)
• Message Passing Neural PDE Solvers [60.77761603258397]
  We build a neural message passing solver, replacing allally designed components in the graph with backprop-optimized neural function approximators. We show that neural message passing solvers representationally contain some classical methods, such as finite differences, finite volumes, and WENO schemes. We validate our method on various fluid-like flow problems, demonstrating fast, stable, and accurate performance across different domain topologies, equation parameters, discretizations, etc., in 1D and 2D.
  arXiv  Detail & Related papers  (2022-02-07T17:47:46Z)
• Non-Gradient Manifold Neural Network [79.44066256794187]
  Deep neural network (DNN) generally takes thousands of iterations to optimize via gradient descent. We propose a novel manifold neural network based on non-gradient optimization.
  arXiv  Detail & Related papers  (2021-06-15T06:39:13Z)
• On the Stability Properties and the Optimization Landscape of Training Problems with Squared Loss for Neural Networks and General Nonlinear Conic Approximation Schemes [0.0]
  We study the optimization landscape and the stability properties of training problems with squared loss for neural networks and general nonlinear conic approximation schemes. We prove that the same effects that are responsible for these instability properties are also the reason for the emergence of saddle points and spurious local minima.
  arXiv  Detail & Related papers  (2020-11-06T11:34:59Z)
• Multipole Graph Neural Operator for Parametric Partial Differential Equations [57.90284928158383]
  One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data. We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity. Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
  arXiv  Detail & Related papers  (2020-06-16T21:56:22Z)
• Neural Control Variates [71.42768823631918]
  We show that a set of neural networks can face the challenge of finding a good approximation of the integrand. We derive a theoretically optimal, variance-minimizing loss function, and propose an alternative, composite loss for stable online training in practice. Specifically, we show that the learned light-field approximation is of sufficient quality for high-order bounces, allowing us to omit the error correction and thereby dramatically reduce the noise at the cost of negligible visible bias.
  arXiv  Detail & Related papers  (2020-06-02T11:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.