Quantitative Gaussian-Process limits of Tensor Programs

• URL: http://arxiv.org/abs/2607.06290v1
• Date: Tue, 07 Jul 2026 13:59:56 GMT
• Title: Quantitative Gaussian-Process limits of Tensor Programs
• Abstract summary: We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs.<n>Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits.
• Score: 3.441021278275805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.

Related papers

• How Deep Are Deep GPs, Really? A Sharp Threshold and a Non-Gaussian Limit for Compositional GPs [48.096969031315744]
  Previous work has established that for the RBF kernel and a certain range of bandwidths $r$, the prior degenerates in the limit.<n>We prove that for $r$ below the threshold $r_c(d)$ the prior converges to a limit distribution $_barZ$.
  arXiv  Detail & Related papers  (2026-06-06T15:12:43Z)
• Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations [42.57657519369106]
  This work establishes rigorous first-of-its-kind upper bounds on the generalization error for the method of approximating solutions to Navier-Stokes equations.<n>We trained depth-2 neural networks trained via the unsupervised Physics-Informed Neural Network framework.<n>We provide empirical validation of the suggested activation functions and the corresponding bounds on a PINN setup solving the Taylor-Green vortex benchmark.
  arXiv  Detail & Related papers  (2026-03-24T11:08:34Z)
• Quantitative convergence of trained single layer neural networks to Gaussian processes [3.441021278275805]
  We study the quantitative convergence of shallow neural networks trained via gradient descent to their associated Gaussian processes in the infinite-width limit.<n>We provide explicit upper bounds on the quadratic Wasserstein distance between the network output and its Gaussian approximation at any training time.<n>Our results quantify how architectural parameters, such as width and input dimension, influence convergence, and how training dynamics affect the approximation error.
  arXiv  Detail & Related papers  (2025-09-29T09:59:27Z)
• Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs [15.559242245579382]
  In this paper, we identify the infinite-width limit distribution of variables within a single attention layer under realistic architectural dimensionality.<n>We derive the exact form of this limit law without resorting to infinite-head approximations or tailored scalings.<n>Our findings lay the groundwork for developing a unified theory of deep Transformer architectures in the infinite-width regime.
  arXiv  Detail & Related papers  (2025-06-01T05:53:47Z)
• Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation [57.10353686244835]
  We address the problem of verifying neural networks against geometric transformations of the input image, including rotation, scaling, shearing, and translation. The proposed method computes provably sound piecewise linear constraints for the pixel values by using sampling and linear approximations in combination with branch-and-bound Lipschitz. We show that our proposed implementation resolves up to 32% more verification cases than present approaches.
  arXiv  Detail & Related papers  (2024-08-23T15:02:09Z)
• Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes [1.0878040851638]
  We show that the distance between the network output and the corresponding Gaussian approximation scales inversely with the width of the network, exhibiting faster convergence than the naive suggested by the central limit theorem. We also apply our bounds to obtain theoretical approximations for the exact posterior distribution of the network, when the likelihood is a bounded Lipschitz function of the network output evaluated on a (finite) training set.
  arXiv  Detail & Related papers  (2023-12-18T22:29:40Z)
• Posterior Inference on Shallow Infinitely Wide Bayesian Neural Networks under Weights with Unbounded Variance [1.5960546024967326]
  It is known that the infinite width scaling limit of a Bayesian neural network with one hidden layer is a Gaussian process, when the network weights have bounded prior variance. Neal's result has been extended to networks with multiple hidden layers and to convolutional neural networks, also with Gaussian process scaling limits. Our contribution is an interpretable and computationally efficient procedure for posterior inference, using a conditionally Gaussian representation, that then allows full use of the Gaussian process machinery for tractable posterior inference and uncertainty quantification in the non-Gaussian regime.
  arXiv  Detail & Related papers  (2023-05-18T02:55:00Z)
• Provable Lipschitz Certification for Generative Models [103.97252161103042]
  We present a scalable technique for upper bounding the Lipschitz constant of generative models. We approximate this set by layerwise convex approximations using zonotopes. This provides efficient and tight bounds on small networks and can scale to generative models on VAE and DCGAN architectures.
  arXiv  Detail & Related papers  (2021-07-06T17:00:29Z)
• Divergence Frontiers for Generative Models: Sample Complexity, Quantization Level, and Frontier Integral [58.434753643798224]
  Divergence frontiers have been proposed as an evaluation framework for generative models. We establish non-asymptotic bounds on the sample complexity of the plug-in estimator of divergence frontiers. We also augment the divergence frontier framework by investigating the statistical performance of smoothed distribution estimators.
  arXiv  Detail & Related papers  (2021-06-15T06:26:25Z)
• Scaling limits of lattice quantum fields by wavelets [62.997667081978825]
  The renormalization group is considered as an inductive system of scaling maps between lattice field algebras. We show that the inductive limit of free lattice ground states exists and the limit state extends to the familiar massive continuum free field.
  arXiv  Detail & Related papers  (2020-10-21T16:30:06Z)
• Asymptotics of Wide Convolutional Neural Networks [18.198962344790377]
  We study scaling laws for wide CNNs and networks with skip connections. We find that the difference in performance between finite and infinite width models vanishes at a definite rate with respect to model width.
  arXiv  Detail & Related papers  (2020-08-19T21:22:19Z)
• Local Propagation in Constraint-based Neural Network [77.37829055999238]
  We study a constraint-based representation of neural network architectures. We investigate a simple optimization procedure that is well suited to fulfil the so-called architectural constraints.
  arXiv  Detail & Related papers  (2020-02-18T16:47:38Z)
• Operator-algebraic renormalization and wavelets [62.997667081978825]
  We construct the continuum free field as the scaling limit of Hamiltonian lattice systems using wavelet theory. A renormalization group step is determined by the scaling equation identifying lattice observables with the continuum field smeared by compactly supported wavelets.
  arXiv  Detail & Related papers  (2020-02-04T18:04:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.