Uniform-in-time concentration in two-layer neural networks via transportation inequalities

• URL: http://arxiv.org/abs/2603.01842v1
• Date: Mon, 02 Mar 2026 13:19:59 GMT
• Title: Uniform-in-time concentration in two-layer neural networks via transportation inequalities
• Abstract summary: We prove uniform-in-time concentration of the empirical parameter measure around its mean-field limit in the Wasserstein distance W 1.<n>We also derive analogous concentration bounds in the sliced-Wasserstein distance SW 1.
• Score: 1.873444918172383
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We quantify, uniformly over time and with high probability, the discrepancy between the predictions of a two-layer neural network trained by stochastic gradient descent (SGD) and their mean-field limit, for quadratic loss and ridge regularization. As a key ingredient, we establish T p transportation inequalities (p $\in$ {1, 2}) for the law of the SGD parameters, with explicit constants independent of the iteration index. We then prove uniform-in-time concentration of the empirical parameter measure around its mean-field limit in the Wasserstein distance W 1 , and we translate these bounds into prediction-error estimates against a fixed test function $Φ$. We also derive analogous concentration bounds in the sliced-Wasserstein distance SW 1 , leading to dimension-free rates.

Related papers

• Uniform-in-time Propagation-of-Chaos for Stein Variational Gradient Descent [16.01291417072387]
  We study uniform-in-time propagation-of-chaos for continuous-time Stein Variational Gradient Descent flows.<n>For broad distributional metrics, we introduce a cutoff strategy which combines finite-time propagation-of-chaos estimates up to an $N$-dependent horizon.<n>We also develop a finite-dimensional theory for matrix-valued finite-rank kernels.
  arXiv  Detail & Related papers  (2026-06-30T20:27:39Z)
• Quantifying Uncertainty In Wide Two-Layer Neural Networks: On The Law Of The Limiting Fluctuation Process [3.0171175820634826]
  Uncertainty quantification in neural networks prediction is a main issue for usual applications.<n>Our approach seeks at reducing costs by directly evaluating uncertainty using PDE's information on the variance.<n>We thus study the law of the limiting process describing the random fluctuations around the mean-field limit of wide two-layer neural networks trained by gradient descent.
  arXiv  Detail & Related papers  (2026-06-04T10:25:23Z)
• Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics [92.39053980710702]
  Gradient Descent (SGD) is commonly modeled as a Langevin process, assuming that minibatch noise acts as Brownian motion.<n>This approximation relies on a continuous-time limit and a sqrt(eta) noise scaling that does not match the discrete SGD update at finite learning rate.<n>We propose an alternative formulation of SGD as deterministic dynamics in a fluctuating loss landscape induced by minibatch sampling.
  arXiv  Detail & Related papers  (2026-05-21T15:50:40Z)
• Decentralized Proximal Stochastic Gradient Langevin Dynamics [4.385194124090593]
  Decentralized Proximal Gradient Langevin Dynamics (DE-PSGLD) for sampling from a log-concave probability distribution constrained to a convex domain.<n>As the first decentralized approach for constrained domains, our algorithm exhibits fast posterior concentration and high predictive accuracy.
  arXiv  Detail & Related papers  (2026-05-01T15:11:06Z)
• Finite-Sample Wasserstein Error Bounds and Concentration Inequalities for Nonlinear Stochastic Approximation [6.800624963330628]
  We derive non-asymptotic error bounds for nonlinear approximation algorithms in the Wasserstein-$p$ distance.<n>We show that the normalized last iterates converge to a Gaussian distribution in the $p$-Wasserstein distance at a rate of order $_n1/6$, where $_n$ is the step size.<n>These distributional guarantees imply high-probability concentration inequalities that improve upon those derived from moment bounds and Markovs inequality.
  arXiv  Detail & Related papers  (2026-02-02T18:41:06Z)
• Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching [60.37045080890305]
  We provide the first analysis of the sample complexity for flow-matching based generative models.<n>We decompose the velocity field estimation error into neural-network approximation error, statistical error due to the finite sample size, and optimization error due to the finite number of optimization steps for estimating the velocity field.
  arXiv  Detail & Related papers  (2025-12-01T05:14:25Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Statistical guarantees for continuous-time policy evaluation: blessing of ellipticity and new tradeoffs [2.926192989090622]
  We study the estimation of the value function for continuous-time Markov diffusion processes.<n>Our work provides non-asymptotic statistical guarantees for the least-squares temporal-difference method.
  arXiv  Detail & Related papers  (2025-02-06T18:39:03Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization [90.87444114491116]
  This paper studies minimax optimization problems defined over infinite-dimensional function classes of overparametricized two-layer neural networks. We address (i) the convergence of the gradient descent-ascent algorithm and (ii) the representation learning of the neural networks. Results show that the feature representation induced by the neural networks is allowed to deviate from the initial one by the magnitude of $O(alpha-1)$, measured in terms of the Wasserstein distance.
  arXiv  Detail & Related papers  (2024-04-18T16:46:08Z)
• Dynamics of correlation spreading in low-dimensional transverse-field Ising models [0.0]
  We investigate the dynamical spreading of correlations after a quantum quench starting from a magnetically disordered state in the transverse-field Ising model at one (1D) and two spatial dimensions (2D) We analyze specifically the longitudinal and transverse spin-spin correlation functions at equal time with use of several methods. Our findings provide useful benchmarks for quantum simulation experiments of correlation spreading and theoretical refinement of the Lieb-Robinson bound in the future.
  arXiv  Detail & Related papers  (2023-01-04T02:02:21Z)
• Controlling Moments with Kernel Stein Discrepancies [74.82363458321939]
  Kernel Stein discrepancies (KSDs) measure the quality of a distributional approximation.<n>We first show that standard KSDs used for weak convergence control fail to control moment convergence.<n>We then provide sufficient conditions under which alternative diffusion KSDs control both moment and weak convergence.
  arXiv  Detail & Related papers  (2022-11-10T08:24:52Z)
• Unbalanced Sobolev Descent [31.777218621726284]
  We introduce Unbalanced Sobolev Descent (USD), a particle descent algorithm for transporting a high dimensional source distribution to a target distribution that does not necessarily have the same mass. USD transports particles along flows of the witness function of the Sobolev-Fisher discrepancy (advection step) and reweighs the mass of particles with respect to this witness function (reaction step) We show on synthetic examples that USD transports distributions with or without conservation of mass faster than previous particle descent algorithms.
  arXiv  Detail & Related papers  (2020-09-29T16:43:38Z)
• A Dynamical Central Limit Theorem for Shallow Neural Networks [48.66103132697071]
  We prove that the fluctuations around the mean limit remain bounded in mean square throughout training. If the mean-field dynamics converges to a measure that interpolates the training data, we prove that the deviation eventually vanishes in the CLT scaling.
  arXiv  Detail & Related papers  (2020-08-21T18:00:50Z)
• Quantitative Propagation of Chaos for SGD in Wide Neural Networks [39.35545193410871]
  In this paper, we investigate the limiting behavior of a continuous-time counterpart of the Gradient Descent (SGD) We show 'propagation of chaos' for the particle system defined by this continuous-time dynamics under different scenarios. We identify two under which different mean-field limits are obtained, one of them corresponding to an implicitly regularized version of the minimization problem at hand.
  arXiv  Detail & Related papers  (2020-07-13T12:55:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.