Quantifying constraint hierarchies in Bayesian PINNs via per-constraint Hessian decomposition

• URL: http://arxiv.org/abs/2510.03278v1
• Date: Sun, 28 Sep 2025 11:06:46 GMT
• Title: Quantifying constraint hierarchies in Bayesian PINNs via per-constraint Hessian decomposition
• Abstract summary: We introduce a scalable, matrix-free Laplace framework that decomposes the posterior Hessian into contributions from each constraint.<n>Our method tracks how constraints sculpt the network's geometry and shows, directly through the Hessian, how changing a single loss weight non-trivially redistributes curvature and effective dominance across the others.
• Score: 0.27214777196418644
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bayesian physics-informed neural networks (B-PINNs) merge data with governing equations to solve differential equations under uncertainty. However, interpreting uncertainty and overconfidence in B-PINNs requires care due to the poorly understood effects the physical constraints have on the network; overconfidence could reflect warranted precision, enforced by the constraints, rather than miscalibration. Motivated by the need to further clarify how individual physical constraints shape these networks, we introduce a scalable, matrix-free Laplace framework that decomposes the posterior Hessian into contributions from each constraint and provides metrics to quantify their relative influence on the loss landscape. Applied to the Van der Pol equation, our method tracks how constraints sculpt the network's geometry and shows, directly through the Hessian, how changing a single loss weight non-trivially redistributes curvature and effective dominance across the others.

Related papers

• Multi-dimensional training-priority weighting based on physical information propagation paths: a unified residual-weighting framework for physics-informed neural networks [7.031909944831412]
  Physics-informed neural networks (PINNs) have shown promise for solving partial differential equations (PDEs)<n>PINNs' synchronous optimization treats residuals of different regions and constraints equally, which is inconsistent with the progressive "from source to response" physical information propagation path, degrading training stability and accuracy.<n>We propose a unified multi-dimensional priority-constraint framework that partitions the domain along the propagation path and constructs negative-exponential residual weights, converting the physical propagation order into a training priority.
  arXiv  Detail & Related papers  (2026-07-13T05:02:07Z)
• Adaptive Hard-Soft Physics-Informed Neural Networks for Robust Boundary-Constrained PDE Solving [0.29360071145551064]
  Physics-informed neural networks (PINNs) solve partial differential equations (PDEs) by embedding physical principles into the learning process.<n>This study proposes a unified hard-soft physics--informed neural network (HSPINN) with adaptive loss weighting.<n>HSPINN consistently achieves faster convergence, higher accuracy, and greater stability than conventional PINNs.
  arXiv  Detail & Related papers  (2026-06-22T13:58:56Z)
• Convolution-weighting method for the physics-informed neural network: A Primal-Dual Optimization Perspective [14.65008276932511]
  Physics-informed neural networks (PINNs) are extensively employed to solve partial differential equations (PDEs)<n>PINNs are typically optimized using a finite set of points, which poses significant challenges in guaranteeing their convergence and accuracy.<n>We propose a new weighting scheme that will adaptively change the weights to the loss functions from isolated points to their continuous neighborhood regions.
  arXiv  Detail & Related papers  (2025-06-24T17:13:51Z)
• Non-Asymptotic Stability and Consistency Guarantees for Physics-Informed Neural Networks via Coercive Operator Analysis [0.0]
  We present a unified theoretical framework for analyzing the stability and consistency of Physics-Informed Neural Networks (PINNs)<n>PINNs approximate solutions to partial differential equations (PDEs) by minimizing residual losses over sampled collocation and boundary points.<n>We formalize both operator-level and variational notions of consistency, proving that residual minimization in Sobolev norms leads to convergence in energy and uniform norms under mild regularity.
  arXiv  Detail & Related papers  (2025-06-16T14:41:15Z)
• Robust Stochastically-Descending Unrolled Networks [85.6993263983062]
  Deep unrolling is an emerging learning-to-optimize method that unrolls a truncated iterative algorithm in the layers of a trainable neural network.<n>We show that convergence guarantees and generalizability of the unrolled networks are still open theoretical problems.<n>We numerically assess unrolled architectures trained under the proposed constraints in two different applications.
  arXiv  Detail & Related papers  (2023-12-25T18:51:23Z)
• Solving Forward and Inverse Problems of Contact Mechanics using Physics-Informed Neural Networks [0.0]
  We deploy PINNs in a mixed-variable formulation enhanced by output transformation to enforce hard and soft constraints. We show that PINNs can serve as pure partial equation (PDE) solver, as data-enhanced forward model, and as fast-to-evaluate surrogate model.
  arXiv  Detail & Related papers  (2023-08-24T11:31:24Z)
• Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks [5.8010446129208155]
  Physics-informed neural networks (PINNs) leverage neural-networks to find the solutions of partial differential equation (PDE)-constrained optimization problems. We show that the challenge of training persists even when the boundary conditions are strictly enforced. We propose reformulating PINNs on a Lagrangian frame of reference, i.e., LPINNs, as a PDE-informed solution.
  arXiv  Detail & Related papers  (2022-05-05T19:48:05Z)
• Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks [83.58049517083138]
  We consider a two-layer ReLU network trained via gradient descent. We show that SGD is biased towards a simple solution. We also provide empirical evidence that knots at locations distinct from the data points might occur.
  arXiv  Detail & Related papers  (2021-11-03T15:14:20Z)
• Robust Learning of Physics Informed Neural Networks [2.86989372262348]
  Physics-informed Neural Networks (PINNs) have been shown to be effective in solving partial differential equations. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE.
  arXiv  Detail & Related papers  (2021-10-26T00:10:57Z)
• On Convergence of Training Loss Without Reaching Stationary Points [62.41370821014218]
  We show that Neural Network weight variables do not converge to stationary points where the gradient the loss function vanishes. We propose a new perspective based on ergodic theory dynamical systems.
  arXiv  Detail & Related papers  (2021-10-12T18:12:23Z)
• Loss Bounds for Approximate Influence-Based Abstraction [81.13024471616417]
  Influence-based abstraction aims to gain leverage by modeling local subproblems together with the 'influence' that the rest of the system exerts on them. This paper investigates the performance of such approaches from a theoretical perspective. We show that neural networks trained with cross entropy are well suited to learn approximate influence representations.
  arXiv  Detail & Related papers  (2020-11-03T15:33:10Z)
• Optimizing Mode Connectivity via Neuron Alignment [84.26606622400423]
  Empirically, the local minima of loss functions can be connected by a learned curve in model space along which the loss remains nearly constant. We propose a more general framework to investigate effect of symmetry on landscape connectivity by accounting for the weight permutations of networks being connected.
  arXiv  Detail & Related papers  (2020-09-05T02:25:23Z)
• Revisiting Initialization of Neural Networks [72.24615341588846]
  We propose a rigorous estimation of the global curvature of weights across layers by approximating and controlling the norm of their Hessian matrix. Our experiments on Word2Vec and the MNIST/CIFAR image classification tasks confirm that tracking the Hessian norm is a useful diagnostic tool.
  arXiv  Detail & Related papers  (2020-04-20T18:12:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.