Quantifying Uncertainty In Wide Two-Layer Neural Networks: On The Law Of The Limiting Fluctuation Process

• URL: http://arxiv.org/abs/2606.05982v1
• Date: Thu, 04 Jun 2026 10:25:23 GMT
• Title: Quantifying Uncertainty In Wide Two-Layer Neural Networks: On The Law Of The Limiting Fluctuation Process
• Abstract summary: Uncertainty quantification in neural networks prediction is a main issue for usual applications.<n>Our approach seeks at reducing costs by directly evaluating uncertainty using PDE's information on the variance.<n>We thus study the law of the limiting process describing the random fluctuations around the mean-field limit of wide two-layer neural networks trained by gradient descent.
• Score: 3.0171175820634826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Uncertainty quantification in neural networks prediction is a main issue for usual applications. Our approach seeks at reducing computation costs by directly evaluating uncertainty using PDE's information on the asymptotic variance, rather than the deep ensemble method which may be seen as a Monte Carlo estimation of the prediction, requiring the training of multiple networks. We thus study the law of the limiting process describing the random fluctuations around the mean-field limit of wide two-layer neural networks trained by stochastic gradient descent in a weak-noise regime. Building on a recent trajectorial central limit theorem, in which this limit is characterized as the weak solution of a linear stochastic evolution equation, we identify its law explicitly. More precisely, we show that it is a centered Gaussian process in the dual of a weighted Sobolev space, and we derive a closed covariance representation for the finite-dimensional distributions obtained by testing it against smooth functions. This covariance is expressed through the solution of a backward transport equation with a nonlocal source term, whose coefficients are driven by the mean-field trajectory. As a consequence, by testing against the activation function at a fixed input, we obtain an expression for the limiting variance of the corresponding network-output fluctuations. We illustrate this result numerically on a one-dimensional regression example.

Related papers

• Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exit [15.784279009967888]
  We study compute reduction in neural networks through a unified partial versus full view.<n>In an magnitude single-neuron model, we prove a concentration theorem for one-shot magnitude pruning with explicit rates.<n>We extend the analysis to deep networks, characterizing how pruning-induced distortions accumulate with depth.
  arXiv  Detail & Related papers  (2026-09-11T01:38:44Z)
• Implicit Variational Rejection Sampling [49.24060583051337]
  Variational Inference (VI) is a fundamental inference technique in Bayesian machine learning for approxing complex posterior distributions.<n>Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility.<n>We propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation.
  arXiv  Detail & Related papers  (2026-06-12T08:17:45Z)
• Physics-informed neural particle flow for the Bayesian update step [0.8220217498103312]
  We propose a physics-informed neural particle flow, which is an amortized inference framework.<n>By embedding a governing partial differential equation (PDE) into the loss function, we train a neural network to approximate the transport velocity field.<n>We demonstrate that the neural parameterization acts as an implicit regularizer, mitigating the stiffness inherent to analytic flows.
  arXiv  Detail & Related papers  (2026-02-26T15:10:45Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• BiLO: Bilevel Local Operator Learning for PDE Inverse Problems. Part II: Efficient Uncertainty Quantification with Low-Rank Adaptation [9.229577043169224]
  Uncertainty quantification and inverse problems governed by partial differential equations (PDEs) are central to a wide range of scientific and engineering applications.<n>We extend Bilevel Local Operator Learning (BiLO) for PDE-constrained optimization problems developed in Part 1 to the Bayesian inference framework.
  arXiv  Detail & Related papers  (2025-07-22T21:20:20Z)
• Ultra-fast feature learning for the training of two-layer neural networks in the two-timescale regime [26.47265060394168]
  We study the convergence of methods for the training of mean-field-hidden-layer neural networks.<n>We show that a strategy enables provable convergence rates for the sampling of a teacher feature distribution.
  arXiv  Detail & Related papers  (2025-04-25T09:40:10Z)
• A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization [90.87444114491116]
  This paper studies minimax optimization problems defined over infinite-dimensional function classes of overparametricized two-layer neural networks. We address (i) the convergence of the gradient descent-ascent algorithm and (ii) the representation learning of the neural networks. Results show that the feature representation induced by the neural networks is allowed to deviate from the initial one by the magnitude of $O(alpha-1)$, measured in terms of the Wasserstein distance.
  arXiv  Detail & Related papers  (2024-04-18T16:46:08Z)
• Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals [3.4240632942024685]
  We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score based MCMC method that is deterministic, resulting in a deterministic evolution for particles.
  arXiv  Detail & Related papers  (2023-08-28T23:51:33Z)
• An application of the splitting-up method for the computation of a neural network representation for the solution for the filtering equations [68.8204255655161]
  Filtering equations play a central role in many real-life applications, including numerical weather prediction, finance and engineering. One of the classical approaches to approximate the solution of the filtering equations is to use a PDE inspired method, called the splitting-up method. We combine this method with a neural network representation to produce an approximation of the unnormalised conditional distribution of the signal process.
  arXiv  Detail & Related papers  (2022-01-10T11:01:36Z)
• Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks [83.58049517083138]
  We consider a two-layer ReLU network trained via gradient descent. We show that SGD is biased towards a simple solution. We also provide empirical evidence that knots at locations distinct from the data points might occur.
  arXiv  Detail & Related papers  (2021-11-03T15:14:20Z)
• A Dynamical Central Limit Theorem for Shallow Neural Networks [48.66103132697071]
  We prove that the fluctuations around the mean limit remain bounded in mean square throughout training. If the mean-field dynamics converges to a measure that interpolates the training data, we prove that the deviation eventually vanishes in the CLT scaling.
  arXiv  Detail & Related papers  (2020-08-21T18:00:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.