Stability Analysis of Physics-Informed Neural Networks via Variational Coercivity, Perturbation Bounds, and Concentration Estimates

• URL: http://arxiv.org/abs/2506.13554v1
• Date: Mon, 16 Jun 2025 14:41:15 GMT
• Title: Stability Analysis of Physics-Informed Neural Networks via Variational Coercivity, Perturbation Bounds, and Concentration Estimates
• Authors: Ronald Katende,
• Abstract summary: PINNs approximate solutions to partial differential equations (PDEs) by minimizing residual-based losses over sampled collocation points.<n>We derive deterministic stability bounds that quantify how bounded perturbations in the network output propagate through both residual and supervised loss components.<n>This work provides a mathematically grounded and practically applicable stability framework for PINNs, clarifying the role of operator structure, sampling design, and functional regularity in robust training.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop a rigorous stability framework for Physics-Informed Neural Networks (PINNs) grounded in variational analysis, operator coercivity, and explicit perturbation theory. PINNs approximate solutions to partial differential equations (PDEs) by minimizing residual-based losses over sampled collocation points. We derive deterministic stability bounds that quantify how bounded perturbations in the network output propagate through both residual and supervised loss components. Probabilistic stability is established via McDiarmid's inequality, yielding non-asymptotic concentration bounds that link sampling variability to empirical loss fluctuations under minimal assumptions. Generalization from Sobolev-norm training loss to uniform approximation is analyzed using coercivity and Sobolev embeddings, leading to pointwise error control. The theoretical results apply to both scalar and vector-valued PDEs and cover composite loss formulations. Numerical experiments validate the perturbation sensitivity, sample complexity estimates, and Sobolev-to-uniform generalization bounds. This work provides a mathematically grounded and practically applicable stability framework for PINNs, clarifying the role of operator structure, sampling design, and functional regularity in robust training.

Related papers

• Causal Operator Discovery in Partial Differential Equations via Counterfactual Physics-Informed Neural Networks [0.0]
  We develop a principled framework for discovering causal structure in partial differential equations (PDEs) using physics-informed neural networks and counterfactual minimizations.<n>We validate the framework on both synthetic and real-world datasets across climate dynamics, tumor diffusion, and ocean flows.<n>This work positions causal PDE discovery as a tractable and interpretable inference task grounded in structural causal models and variational residual analysis.
  arXiv  Detail & Related papers  (2025-06-25T07:15:42Z)
• Wasserstein Distributionally Robust Nonparametric Regression [9.65010022854885]
  This paper studies the generalization properties of Wasserstein distributionally robust nonparametric estimators.<n>We establish non-asymptotic error bounds for the excess local worst-case risk.<n>The robustness of the proposed estimator is evaluated through simulation studies and illustrated with an application to the MNIST dataset.
  arXiv  Detail & Related papers  (2025-05-12T18:07:37Z)
• Unified theoretical guarantees for stability, consistency, and convergence in neural PDE solvers from non-IID data to physics-informed networks [0.0]
  We establish a unified theoretical framework addressing the stability, consistency, and convergence of neural networks under realistic training conditions.<n>For standard supervised learning with dependent data, we derive uniform stability bounds for gradient-based methods.<n>In federated learning with heterogeneous data, we quantify model inconsistency via curvature-aware aggregation and information-theoretic divergence.
  arXiv  Detail & Related papers  (2024-09-08T08:48:42Z)
• Stable Nonconvex-Nonconcave Training via Linear Interpolation [51.668052890249726]
  This paper presents a theoretical analysis of linearahead as a principled method for stabilizing (large-scale) neural network training. We argue that instabilities in the optimization process are often caused by the nonmonotonicity of the loss landscape and show how linear can help by leveraging the theory of nonexpansive operators.
  arXiv  Detail & Related papers  (2023-10-20T12:45:12Z)
• Learning Discretized Neural Networks under Ricci Flow [48.47315844022283]
  We study Discretized Neural Networks (DNNs) composed of low-precision weights and activations.<n>DNNs suffer from either infinite or zero gradients due to the non-differentiable discrete function during training.
  arXiv  Detail & Related papers  (2023-02-07T10:51:53Z)
• Tunable Complexity Benchmarks for Evaluating Physics-Informed Neural Networks on Coupled Ordinary Differential Equations [64.78260098263489]
  In this work, we assess the ability of physics-informed neural networks (PINNs) to solve increasingly-complex coupled ordinary differential equations (ODEs) We show that PINNs eventually fail to produce correct solutions to these benchmarks as their complexity increases. We identify several reasons why this may be the case, including insufficient network capacity, poor conditioning of the ODEs, and high local curvature, as measured by the Laplacian of the PINN loss.
  arXiv  Detail & Related papers  (2022-10-14T15:01:32Z)
• A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks [12.355137704908042]
  We show restrained numerical instabilities in current training practices of deep networks with gradient descent (SGD) We do this by presenting a theoretical framework using numerical analysis of partial differential equations (PDE), and analyzing the gradient descent PDE of convolutional neural networks (CNNs) We show this is a consequence of the non-linear PDE associated with the descent of the CNN, whose local linearization changes when over-driving the step size of the discretization resulting in a stabilizing effect.
  arXiv  Detail & Related papers  (2022-06-04T14:54:05Z)
• coVariance Neural Networks [119.45320143101381]
  Graph neural networks (GNN) are an effective framework that exploit inter-relationships within graph-structured data for learning. We propose a GNN architecture, called coVariance neural network (VNN), that operates on sample covariance matrices as graphs. We show that VNN performance is indeed more stable than PCA-based statistical approaches.
  arXiv  Detail & Related papers  (2022-05-31T15:04:43Z)
• On Convergence of Training Loss Without Reaching Stationary Points [62.41370821014218]
  We show that Neural Network weight variables do not converge to stationary points where the gradient the loss function vanishes. We propose a new perspective based on ergodic theory dynamical systems.
  arXiv  Detail & Related papers  (2021-10-12T18:12:23Z)
• Stability of Neural Networks on Manifolds to Relative Perturbations [118.84154142918214]
  Graph Neural Networks (GNNs) show impressive performance in many practical scenarios. GNNs can scale well on large size graphs, but this is contradicted by the fact that existing stability bounds grow with the number of nodes.
  arXiv  Detail & Related papers  (2021-10-10T04:37:19Z)
• On the Stability Properties and the Optimization Landscape of Training Problems with Squared Loss for Neural Networks and General Nonlinear Conic Approximation Schemes [0.0]
  We study the optimization landscape and the stability properties of training problems with squared loss for neural networks and general nonlinear conic approximation schemes. We prove that the same effects that are responsible for these instability properties are also the reason for the emergence of saddle points and spurious local minima.
  arXiv  Detail & Related papers  (2020-11-06T11:34:59Z)
• Neural Control Variates [71.42768823631918]
  We show that a set of neural networks can face the challenge of finding a good approximation of the integrand. We derive a theoretically optimal, variance-minimizing loss function, and propose an alternative, composite loss for stable online training in practice. Specifically, we show that the learned light-field approximation is of sufficient quality for high-order bounces, allowing us to omit the error correction and thereby dramatically reduce the noise at the cost of negligible visible bias.
  arXiv  Detail & Related papers  (2020-06-02T11:17:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.