Phase Transitions in the Fluctuations of Functionals of Random Neural Networks

• URL: http://arxiv.org/abs/2604.19738v2
• Date: Thu, 23 Apr 2026 09:10:27 GMT
• Title: Phase Transitions in the Fluctuations of Functionals of Random Neural Networks
• Abstract summary: We establish central and non-central limit theorems for sequences of functionals of an infinitely-wide random neural network on the d-dimensional sphere.<n>We show that the behaviour of these functionals as the depth of the network increases depends crucially on the fixed points of the covariance function.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We establish central and non-central limit theorems for sequences of functionals of the Gaussian output of an infinitely-wide random neural network on the d-dimensional sphere . We show that the asymptotic behaviour of these functionals as the depth of the network increases depends crucially on the fixed points of the covariance function, resulting in three distinct limiting regimes: convergence to the same functional of a limiting Gaussian field, convergence to a Gaussian distribution, convergence to a distribution in the Qth Wiener chaos. Our proofs exploit tools that are now classical (Hermite expansions, Diagram Formula, Stein-Malliavin techniques), but also ideas which have never been used in similar contexts: in particular, the asymptotic behaviour is determined by the fixed-point structure of the iterative operator associated with the covariance, whose nature and stability governs the different limiting regimes.

Related papers

• Quantifying Uncertainty In Wide Two-Layer Neural Networks: On The Law Of The Limiting Fluctuation Process [3.0171175820634826]
  Uncertainty quantification in neural networks prediction is a main issue for usual applications.<n>Our approach seeks at reducing costs by directly evaluating uncertainty using PDE's information on the variance.<n>We thus study the law of the limiting process describing the random fluctuations around the mean-field limit of wide two-layer neural networks trained by gradient descent.
  arXiv  Detail & Related papers  (2026-06-04T10:25:23Z)
• GRIFDIR: Graph Resolution-Invariant FEM Diffusion Models in Function Spaces over Irregular Domains [56.121725064621295]
  Score-based diffusion models in function spaces offer a principled framework for modelling function-valued data.<n>However, practical implementations have struggled to fully realise these benefits.<n>We propose a novel architecture that represents generalised convolutional kernels as finite graph kernels.
  arXiv  Detail & Related papers  (2026-05-05T08:33:52Z)
• Approximation Capabilities of Feedforward Neural Networks with GELU Activations [6.488575826304024]
  We derive an approximation error bound that holds simultaneously for a function and all its derivatives up to any prescribed order.<n>The bounds apply to elementary functions, including multivariates, the exponential function, and the reciprocal function.<n>We report the network size, weight magnitudes, and behavior at infinity.
  arXiv  Detail & Related papers  (2025-12-25T17:56:44Z)
• Approximation properties of neural ODEs [5.828989070109041]
  We prove the universal approximation property (UAP) of shallow neural networks in the space of continuous functions.<n>In particular, we constrain the Lipschitz constant of the neural ODE's flow map and the norms of the weights to increase the network's stability.
  arXiv  Detail & Related papers  (2025-03-19T21:11:28Z)
• Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes [1.0878040851638]
  We show that the distance between the network output and the corresponding Gaussian approximation scales inversely with the width of the network, exhibiting faster convergence than the naive suggested by the central limit theorem. We also apply our bounds to obtain theoretical approximations for the exact posterior distribution of the network, when the likelihood is a bounded Lipschitz function of the network output evaluated on a (finite) training set.
  arXiv  Detail & Related papers  (2023-12-18T22:29:40Z)
• On the relationship between multivariate splines and infinitely-wide neural networks [0.0]
  We show that the associated function space is a Sobolev space on a Euclidean ball, with an explicit bound on the norms of derivatives. This random feature expansion is numerically better behaved than usual random Fourier features, both in theory and practice. In particular, in one dimension, we compare the associated leverage scores to compare the two random expansions and show a better scaling for the neural network expansion.
  arXiv  Detail & Related papers  (2023-02-07T13:29:06Z)
• A Functional-Space Mean-Field Theory of Partially-Trained Three-Layer Neural Networks [49.870593940818715]
  We study the infinite-width limit of a type of three-layer NN model whose first layer is random and fixed. Our theory accommodates different scaling choices of the model, resulting in two regimes of the MF limit that demonstrate distinctive behaviors.
  arXiv  Detail & Related papers  (2022-10-28T17:26:27Z)
• Decimation technique for open quantum systems: a case study with driven-dissipative bosonic chains [62.997667081978825]
  Unavoidable coupling of quantum systems to external degrees of freedom leads to dissipative (non-unitary) dynamics. We introduce a method to deal with these systems based on the calculation of (dissipative) lattice Green's function. We illustrate the power of this method with several examples of driven-dissipative bosonic chains of increasing complexity.
  arXiv  Detail & Related papers  (2022-02-15T19:00:09Z)
• Robust Estimation for Nonparametric Families via Generative Adversarial Networks [92.64483100338724]
  We provide a framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems. Our work extend these to robust mean estimation, second moment estimation, and robust linear regression. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance.
  arXiv  Detail & Related papers  (2022-02-02T20:11:33Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Non-asymptotic approximations of neural networks by Gaussian processes [7.56714041729893]
  We study the extent to which wide neural networks may be approximated by Gaussian processes when with random weights. As the width of a network goes to infinity, its law converges to that of a Gaussian process.
  arXiv  Detail & Related papers  (2021-02-17T10:19:26Z)
• Scaling limits of lattice quantum fields by wavelets [62.997667081978825]
  The renormalization group is considered as an inductive system of scaling maps between lattice field algebras. We show that the inductive limit of free lattice ground states exists and the limit state extends to the familiar massive continuum free field.
  arXiv  Detail & Related papers  (2020-10-21T16:30:06Z)
• A Dynamical Central Limit Theorem for Shallow Neural Networks [48.66103132697071]
  We prove that the fluctuations around the mean limit remain bounded in mean square throughout training. If the mean-field dynamics converges to a measure that interpolates the training data, we prove that the deviation eventually vanishes in the CLT scaling.
  arXiv  Detail & Related papers  (2020-08-21T18:00:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.