Unified generalization analysis for physics informed neural networks

• URL: http://arxiv.org/abs/2605.13260v1
• Date: Wed, 13 May 2026 09:42:33 GMT
• Title: Unified generalization analysis for physics informed neural networks
• Abstract summary: We derive bounds for neural networks that involve differentiation with respect to input variables, covering PINNs and VPINNs under a unified framework.<n>We show that the nonlinearity of the differential operator exponentially enlarges the bound, highlighting its significant impact on generalization.
• Score: 23.137880798964165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physics-Informed Neural Networks (PINNs) and their variational counterparts (VPINNs) are neural networks that incorporate physical laws, making them useful for scientific problems. Existing generalization analyses for PINNs and VPINNs remain limited, often requiring restrictive assumptions such as stability conditions or linear ellipticity. In this paper, we derive generalization bounds for neural networks that involve differentiation with respect to input variables, covering PINNs and VPINNs under a unified framework. We apply Taylor expansion to represent nonlinear differential operators as linear operators on a high-dimensional space, enabling the use of Koopman-based analysis and showing that high-rank networks can generalize well even in settings involving differential operators. We also show that the nonlinearity of the differential operator exponentially enlarges the bound, highlighting its significant impact on generalization.

Related papers

• Learning with Covariance Matrices: Principal Component Analysis Meets Learning with Graphs [94.85514873380572]
  This feature article provides an overview of the theoretical foundations for coVariance neural networks (VNNs)<n>Existing theoretical contributions on graph neural networks (GNNs) consider abstract graph representations.<n>This tutorial brings into focus various novel theoretical insights via mathematical analyses of VNNs that have broad signal processing implications.
  arXiv  Detail & Related papers  (2026-09-09T17:28:46Z)
• Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations [42.57657519369106]
  This work establishes rigorous first-of-its-kind upper bounds on the generalization error for the method of approximating solutions to Navier-Stokes equations.<n>We trained depth-2 neural networks trained via the unsupervised Physics-Informed Neural Network framework.<n>We provide empirical validation of the suggested activation functions and the corresponding bounds on a PINN setup solving the Taylor-Green vortex benchmark.
  arXiv  Detail & Related papers  (2026-03-24T11:08:34Z)
• Benign Overfitting in Deep Neural Networks under Lazy Training [72.28294823115502]
  We show that when the data distribution is well-separated, DNNs can achieve Bayes-optimal test error for classification. Our results indicate that interpolating with smoother functions leads to better generalization.
  arXiv  Detail & Related papers  (2023-05-30T19:37:44Z)
• Resolution-Invariant Image Classification based on Fourier Neural Operators [1.3190581566723918]
  We investigate the use of generalization Neural Operators (FNOs) for image classification in comparison to standard Convolutional Neural Networks (CNNs) We derive the FNO architecture as an example for continuous and Fr'echet-differentiable neural operators on Lebesgue spaces.
  arXiv  Detail & Related papers  (2023-04-02T10:23:36Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• How You Start Matters for Generalization [26.74340246715699]
  We show that the generalization of neural networks is heavily tied to their initializes. We make a case against the controversial flat-minima conjecture.
  arXiv  Detail & Related papers  (2022-06-17T05:30:56Z)
• The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks [3.3946853660795893]
  We derive the infinite-time training limit of a mathematically tractable class of deep nonlinear neural networks (GLNs) We show how architectural constraints and the implicit bias of gradient descent affect performance. By making the inductive bias explicit, our framework is poised to inform the development of more efficient, biologically plausible, and robust learning algorithms.
  arXiv  Detail & Related papers  (2022-02-05T22:37:39Z)
• Deep Neural Networks and PIDE discretizations [2.4063592468412276]
  We propose neural networks that tackle the problems of stability and field-of-view of a Convolutional Neural Network (CNN) We propose integral-based spatially nonlocal operators which are related to global weighted Laplacian, fractional Laplacian and fractional inverse Laplacian operators. We test the effectiveness of the proposed neural architectures on benchmark image classification datasets and semantic segmentation tasks in autonomous driving.
  arXiv  Detail & Related papers  (2021-08-05T08:03:01Z)
• Formalizing Generalization and Robustness of Neural Networks to Weight Perturbations [58.731070632586594]
  We provide the first formal analysis for feed-forward neural networks with non-negative monotone activation functions against weight perturbations. We also design a new theory-driven loss function for training generalizable and robust neural networks against weight perturbations.
  arXiv  Detail & Related papers  (2021-03-03T06:17:03Z)
• Stability of Algebraic Neural Networks to Small Perturbations [179.55535781816343]
  Algebraic neural networks (AlgNNs) are composed of a cascade of layers each one associated to and algebraic signal model. We show how any architecture that uses a formal notion of convolution can be stable beyond particular choices of the shift operator.
  arXiv  Detail & Related papers  (2020-10-22T09:10:16Z)
• Generalization bound of globally optimal non-convex neural network training: Transportation map estimation by infinite dimensional Langevin dynamics [50.83356836818667]
  We introduce a new theoretical framework to analyze deep learning optimization with connection to its generalization error. Existing frameworks such as mean field theory and neural tangent kernel theory for neural network optimization analysis typically require taking limit of infinite width of the network to show its global convergence.
  arXiv  Detail & Related papers  (2020-07-11T18:19:50Z)
• Understanding Generalization in Deep Learning via Tensor Methods [53.808840694241]
  We advance the understanding of the relations between the network's architecture and its generalizability from the compression perspective. We propose a series of intuitive, data-dependent and easily-measurable properties that tightly characterize the compressibility and generalizability of neural networks.
  arXiv  Detail & Related papers  (2020-01-14T22:26:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.