Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputs

• URL: http://arxiv.org/abs/2605.24072v1
• Date: Fri, 22 May 2026 12:45:17 GMT
• Title: Optimal Non-Asymptotic Edgeworth Expansions for Multivariate Neural Network Outputs
• Abstract summary: We approximate deviations for a neural network evaluated in a finite number of inputs.<n>As an application, we quantify error in Bayesian posterior distributions when the prior is replaced by its Edgeworth expansion.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Finite-width fully connected neural networks with Gaussian-initialized weights deviate from their infinite-width Gaussian limit, exhibiting non-vanishing higher-order cumulants. We approximate these deviations, for a neural network evaluated in a finite number of inputs, using multidimensional Edgeworth expansions of arbitrary order $4m-1$, with $m\in\mathbb{N}$. Assuming that the corresponding Gaussian limit has an invertible covariance matrix and that the activation function is polynomially bounded, we establish a bound of order $n^{-m}$ on the total variation distance between the law of the true network output and its Edgeworth approximation, with matching lower bounds. As an application, we quantify the error in Bayesian posterior distributions when the prior is replaced by its Edgeworth expansion. Our results are more general and also apply to sequences of conditionally Gaussian vectors converging to a Gaussian vector with invertible covariance.

Related papers

• How Deep Are Deep GPs, Really? A Sharp Threshold and a Non-Gaussian Limit for Compositional GPs [48.096969031315744]
  Previous work has established that for the RBF kernel and a certain range of bandwidths $r$, the prior degenerates in the limit.<n>We prove that for $r$ below the threshold $r_c(d)$ the prior converges to a limit distribution $_barZ$.
  arXiv  Detail & Related papers  (2026-06-06T15:12:43Z)
• Approximation Capabilities of Feedforward Neural Networks with GELU Activations [6.488575826304024]
  We derive an approximation error bound that holds simultaneously for a function and all its derivatives up to any prescribed order.<n>The bounds apply to elementary functions, including multivariates, the exponential function, and the reciprocal function.<n>We report the network size, weight magnitudes, and behavior at infinity.
  arXiv  Detail & Related papers  (2025-12-25T17:56:44Z)
• Approximation Bounds for Transformer Networks with Application to Regression [9.549045683389085]
  We explore the approximation capabilities of Transformer networks for H"older and Sobolev functions.<n>We establish novel upper bounds for standard Transformer networks approxing sequence-to-sequence mappings.<n>We show that if the self-attention layer in a Transformer can perform column averaging, the network can approximate sequence-to-sequence H"older functions.
  arXiv  Detail & Related papers  (2025-04-16T15:25:58Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Tensor cumulants for statistical inference on invariant distributions [49.80012009682584]
  We show that PCA becomes computationally hard at a critical value of the signal's magnitude. We define a new set of objects, which provide an explicit, near-orthogonal basis for invariants of a given degree. It also lets us analyze a new problem of distinguishing between different ensembles.
  arXiv  Detail & Related papers  (2024-04-29T14:33:24Z)
• Wide Deep Neural Networks with Gaussian Weights are Very Close to Gaussian Processes [1.0878040851638]
  We show that the distance between the network output and the corresponding Gaussian approximation scales inversely with the width of the network, exhibiting faster convergence than the naive suggested by the central limit theorem. We also apply our bounds to obtain theoretical approximations for the exact posterior distribution of the network, when the likelihood is a bounded Lipschitz function of the network output evaluated on a (finite) training set.
  arXiv  Detail & Related papers  (2023-12-18T22:29:40Z)
• Quantitative CLTs in Deep Neural Networks [12.845031126178593]
  We study the distribution of a fully connected neural network with random Gaussian weights and biases. We obtain quantitative bounds on normal approximations valid at large but finite $n$ and any fixed network depth. Our bounds are strictly stronger in terms of their dependence on network width than any previously available in the literature.
  arXiv  Detail & Related papers  (2023-07-12T11:35:37Z)
• Gaussian random field approximation via Stein's method with applications to wide random neural networks [20.554836643156726]
  We develop a novel Gaussian smoothing technique that allows us to transfer a bound in a smoother metric to the $W_$ distance. We obtain the first bounds on the Gaussian random field approximation of wide random neural networks. Our bounds are explicitly expressed in terms of the widths of the network and moments of the random weights.
  arXiv  Detail & Related papers  (2023-06-28T15:35:10Z)
• On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias [50.84569563188485]
  We show that gradient flow converges in direction when labels are determined by the sign of a target network with $r$ neurons. Our result may already hold for mild over- parameterization, where the width is $tildemathcalO(r)$ and independent of the sample size.
  arXiv  Detail & Related papers  (2022-05-18T16:57:10Z)
• Spectral clustering under degree heterogeneity: a case for the random walk Laplacian [83.79286663107845]
  This paper shows that graph spectral embedding using the random walk Laplacian produces vector representations which are completely corrected for node degree. In the special case of a degree-corrected block model, the embedding concentrates about K distinct points, representing communities.
  arXiv  Detail & Related papers  (2021-05-03T16:36:27Z)
• On Linear Stochastic Approximation: Fine-grained Polyak-Ruppert and Non-Asymptotic Concentration [115.1954841020189]
  We study the inequality and non-asymptotic properties of approximation procedures with Polyak-Ruppert averaging. We prove a central limit theorem (CLT) for the averaged iterates with fixed step size and number of iterations going to infinity.
  arXiv  Detail & Related papers  (2020-04-09T17:54:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.