Multi-Preconditioned LBFGS for Training Finite-Basis PINNs

• URL: http://arxiv.org/abs/2601.08709v1
• Date: Tue, 13 Jan 2026 16:38:15 GMT
• Title: Multi-Preconditioned LBFGS for Training Finite-Basis PINNs
• Authors: Marc Salvadó-Benasco, Aymane Kssim, Alexander Heinlein, Rolf Krause, Serge Gratton, Alena Kopaničáková,
• Abstract summary: The MP-LBFGS algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs)<n>A key feature is a novel nonlinear multi-preconditioning mechanism, in which subdomain corrections are optimally combined through the solution of a low-dimensional subspace minimization problem.<n> Numerical experiments indicate that MP-LBFGS can improve convergence speed, as well as model accuracy over standard LBFGS while incurring lower communication overhead.
• Score: 35.66877569643008
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear additive Schwarz method and exploits the domain-decomposition-inspired additive architecture of FBPINNs, in which local neural networks are defined on subdomains, thereby localizing the network representation. Parallel, subdomain-local quasi-Newton corrections are then constructed on the corresponding local parts of the architecture. A key feature is a novel nonlinear multi-preconditioning mechanism, in which subdomain corrections are optimally combined through the solution of a low-dimensional subspace minimization problem. Numerical experiments indicate that MP-LBFGS can improve convergence speed, as well as model accuracy over standard LBFGS while incurring lower communication overhead.

Related papers

• Local adapt-then-combine algorithms for distributed nonsmooth optimization: Achieving provable communication acceleration [50.67878993903822]
  We propose a communication-efficient Adapt-Then-Combine (ATC) framework, FlexATC, unifying numerous ATC-based distributed algorithms.<n>We show for the first time that local updates provably lead to communication acceleration for ATC-based distributed algorithms.
  arXiv  Detail & Related papers  (2026-02-18T02:47:05Z)
• AB-PINNs: Adaptive-Basis Physics-Informed Neural Networks for Residual-Driven Domain Decomposition [3.0307211002269443]
  We introduce adaptive-basis physics-informed neural networks (AB-PINNs)<n>AB-PINNs dynamically adapt to the intrinsic features of the unknown solution.<n>We present comprehensive numerical results which demonstrate the effectiveness of AB-PINNs at solving a variety of complex partial differential equations.
  arXiv  Detail & Related papers  (2025-10-10T02:13:06Z)
• From Sublinear to Linear: Fast Convergence in Deep Networks via Locally Polyak-Lojasiewicz Regions [0.0]
  This paper presents a theoretical challenge on the non-GD same loss of deep neural networks (DNNs)<n>We show that the gradient lower-bounds of the suboptimality gap prove that properly finite-width networks admit a squared gradient within an L.<n>Our work provides a theoretical explanation for the efficiency of the efficiency of deep learning.
  arXiv  Detail & Related papers  (2025-07-29T01:49:16Z)
• Accelerating Natural Gradient Descent for PINNs with Randomized Nyström Preconditioning [0.0]
  Natural Descent Gradient (NGD) is an optimization algorithm for training neural network-based solvers for partial differential equations (PDEs)<n>NGD is often limited by the high computational cost of solving linear systems involving the Gramian matrix.<n>We extend matrix-free NGD to broader classes of problems than previously considered and propose the use of Nystr"om preconditioning to accelerate convergence of the inner CG solver.
  arXiv  Detail & Related papers  (2025-05-16T19:00:40Z)
• RoPINN: Region Optimized Physics-Informed Neural Networks [66.38369833561039]
  Physics-informed neural networks (PINNs) have been widely applied to solve partial differential equations (PDEs) This paper proposes and theoretically studies a new training paradigm as region optimization. A practical training algorithm, Region Optimized PINN (RoPINN), is seamlessly derived from this new paradigm.
  arXiv  Detail & Related papers  (2024-05-23T09:45:57Z)
• Neural Network with Local Converging Input (NNLCI) for Supersonic Flow Problems with Unstructured Grids [0.9152133607343995]
  We develop a neural network with local converging input (NNLCI) for high-fidelity prediction using unstructured data. As a validation case, the NNLCI method is applied to study inviscid supersonic flows in channels with bumps.
  arXiv  Detail & Related papers  (2023-10-23T19:03:37Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Optimization Guarantees of Unfolded ISTA and ADMM Networks With Smooth Soft-Thresholding [57.71603937699949]
  We study optimization guarantees, i.e., achieving near-zero training loss with the increase in the number of learning epochs. We show that the threshold on the number of training samples increases with the increase in the network width.
  arXiv  Detail & Related papers  (2023-09-12T13:03:47Z)
• Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies [1.8434042562191815]
  We introduce additive and multiplicative preconditioning strategies for the widely used L-BFGS. We demonstrate that both additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS.
  arXiv  Detail & Related papers  (2023-06-30T13:35:09Z)
• Neural Basis Functions for Accelerating Solutions to High Mach Euler Equations [63.8376359764052]
  We propose an approach to solving partial differential equations (PDEs) using a set of neural networks. We regress a set of neural networks onto a reduced order Proper Orthogonal Decomposition (POD) basis. These networks are then used in combination with a branch network that ingests the parameters of the prescribed PDE to compute a reduced order approximation to the PDE.
  arXiv  Detail & Related papers  (2022-08-02T18:27:13Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Local Propagation in Constraint-based Neural Network [77.37829055999238]
  We study a constraint-based representation of neural network architectures. We investigate a simple optimization procedure that is well suited to fulfil the so-called architectural constraints.
  arXiv  Detail & Related papers  (2020-02-18T16:47:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.