A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning

• URL: http://arxiv.org/abs/2605.26341v1
• Date: Mon, 25 May 2026 21:30:39 GMT
• Title: A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning
• Abstract summary: We develop a PAC-Bayesian framework for physics-informed machine learning (PIML)<n>We provide high-probability generalisation guarantees in the presence of unbounded losses.<n>Our results provide a principled statistical foundation for the generalisation of physics-informed models.
• Score: 14.207412457685313
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Physics-informed machine learning (PIML) integrates mechanistic knowledge, typically in the form of partial differential equations (PDE), into data-driven models. Despite strong empirical performance, its statistical generalisation properties remain poorly understood, particularly in the regression setting with unbounded losses. Existing analyses rely on approximation or stability arguments and do not fully capture how physical structure influences generalisation from finite data. In this work, we develop a PAC-Bayesian framework for PIML that provides high-probability generalisation guarantees in the presence of unbounded losses. We adopt a multi-task perspective that jointly treats data fidelity, PDE residuals, initial and boundary conditions, avoiding the looseness induced by standard union-bound approaches. Our analysis leverages the structure of physics-informed objectives to derive novel bounds where the complexity scales with input-gradient norms of the losses, revealing a direct link between physical regularity and generalisation. We instantiate this framework under Sobolev and Poincaré-type assumptions, yielding two classes of bounds that trade off statistical complexity and smoothness in different regimes. Building on these results, we propose a self-bounding-aware learning algorithm that directly optimises tractable surrogates of the derived bounds, along with a practical procedure to estimate the associated constants in realistic settings. Empirical evaluations on standard PDE benchmarks demonstrate that our bounds are non-vacuous, significantly tighter than union-bound baselines, and can be effectively minimised during training. Overall, our results provide a principled statistical foundation for the generalisation of physics-informed models.

Related papers

• On the Structural Limits of Machine Learning Decision Systems: An Information-Theoretic, Interaction-Based, and Stochastic-Dynamical Perspective [0.0]
  In this work we examine intrinsic limits of data-driven decision systems from an information-theoretic and interaction-based perspective.<n>We analyze minimal achievable error in classification through Fano-type bounds and precision limits in parametric estimation via the Cramér-Rao inequality.
  arXiv  Detail & Related papers  (2026-08-13T17:34:41Z)
• Non-Parametric Rehearsal Learning via Conditional Mean Embeddings [88.89267783967263]
  We propose a non-parametric rehearsal learning approach for textitavoiding undesired future (AUF) problems.<n>Specifically, we use kernel machinery to reformulate the AUF objective into a unified representation that disentangles desirability modeling from action-induced distributional changes.
  arXiv  Detail & Related papers  (2026-05-09T15:30:52Z)
• Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle [51.56484100374058]
  This work proposes that information can be defined as the deviation $$ of realized dynamics from a constrained maximum-caliber (MaxCal) path ensemble over a finite time horizon.<n>Under this definition, each of the cause/effect repertoires central to IIT 3.0 emerge directly from MaxCal variational principles.<n>Information $$ is shown to be equivalent to prediction error under accompanying predictive coding models.
  arXiv  Detail & Related papers  (2026-05-03T07:22:55Z)
• PAC-Bayes Bounds for Gibbs Posteriors via Singular Learning Theory [10.93258787701145]
  We derive explicit non-asymptotic PAC-Bayes generalization bounds for Gibbs posteriors.<n>Unlike classical worst-case complexity bounds based on uniform laws of large numbers, our analysis yields posterior-averaged risk bounds.
  arXiv  Detail & Related papers  (2026-04-19T03:00:18Z)
• Towards A Unified PAC-Bayesian Framework for Norm-based Generalization Bounds [63.47271262149291]
  We propose a unified framework for PAC-Bayesian norm-based generalization.<n>The key to our approach is a sensitivity matrix that quantifies the network outputs with respect to structured weight perturbations.<n>We derive a family of generalization bounds that recover several existing PAC-Bayesian results as special cases.
  arXiv  Detail & Related papers  (2026-01-13T00:42:22Z)
• The Procrustean Bed of Time Series: The Optimization Bias of Point-wise Loss [53.542743390809356]
  This paper aims to provide a first-principles analysis of the Expectation of Optimization Bias (EOB)<n>Our analysis reveals a fundamental paradigm paradox: the more deterministic and structured the time series, the more severe the bias by point-wise loss function.<n>We present a concrete solution that simultaneously achieves both principles via DFT or DWT.
  arXiv  Detail & Related papers  (2025-12-21T06:08:22Z)
• Physics-Embedded Gaussian Process for Traffic State Estimation [15.0051892692122]
  Physical models have difficulty integrating uncertainties and capturing the real complexity of traffic.<n>PEGP is designed to integrate domain knowledge with data-driven methods in traffic state estimation.<n> Experiments on HighD, NGSIM show consistent improvements over non-physics baselines.
  arXiv  Detail & Related papers  (2025-12-03T17:43:40Z)
• Mutual Information Free Topological Generalization Bounds via Stability [46.63069403118614]
  We introduce a novel learning theoretic framework that departs from the existing strategies.<n>We prove that the generalization error of trajectory-stable algorithms can be upper bounded in terms of TDA terms.
  arXiv  Detail & Related papers  (2025-07-09T12:03:25Z)
• Non-Asymptotic Stability and Consistency Guarantees for Physics-Informed Neural Networks via Coercive Operator Analysis [0.0]
  We present a unified theoretical framework for analyzing the stability and consistency of Physics-Informed Neural Networks (PINNs)<n>PINNs approximate solutions to partial differential equations (PDEs) by minimizing residual losses over sampled collocation and boundary points.<n>We formalize both operator-level and variational notions of consistency, proving that residual minimization in Sobolev norms leads to convergence in energy and uniform norms under mild regularity.
  arXiv  Detail & Related papers  (2025-06-16T14:41:15Z)
• Paving the way for scientific foundation models: enhancing generalization and robustness in PDEs with constraint-aware pre-training [49.8035317670223]
  A scientific foundation model (SciFM) is emerging as a promising tool for learning transferable representations across diverse domains.<n>We propose incorporating PDE residuals into pre-training either as the sole learning signal or in combination with data loss to compensate for limited or infeasible training data.<n>Our results show that pre-training with PDE constraints significantly enhances generalization, outperforming models trained solely on solution data.
  arXiv  Detail & Related papers  (2025-03-24T19:12:39Z)
• Generalization Bounds of Surrogate Policies for Combinatorial Optimization Problems [53.03951222945921]
  We analyze smoothed (perturbed) policies, adding controlled random perturbations to the direction used by the linear oracle.<n>Our main contribution is a generalization bound that decomposes the excess risk into perturbation bias, statistical estimation error, and optimization error.<n>We illustrate the scope of the results on applications such as vehicle scheduling, highlighting how smoothing enables both tractable training and controlled generalization.
  arXiv  Detail & Related papers  (2024-07-24T12:00:30Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.