Avoiding unsafe sets when training with Langevin Dynamics

• URL: http://arxiv.org/abs/2607.07538v2
• Date: Wed, 15 Jul 2026 17:21:26 GMT
• Title: Avoiding unsafe sets when training with Langevin Dynamics
• Abstract summary: We study the shape-free bound $_t(mathcalA_H) le (mathcalA_H) le (mathcalA_H) le (mathcalA_H) le (mathcalA_H) le (mathcalA_H) le (mathcalA_H) le (mathcalA_H)
• Score: 1.0762008415887194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Training a model with noisy gradient descent can be idealized as overdamped Langevin dynamics, and a natural safety question is to bound the probability $ν_t(\mathcal{A}_H) = \mathbb{P}(Q_t \in \mathcal{A}_H)$ that the trajectory lies in a designated failure region $\mathcal{A}_H$. We study this for a smooth, strongly convex loss in $d$ dimensions, with $\mathcal{A}_H$ separated from the minimizer by an energy gap. At the end of training, the equilibrium mass $π(\mathcal{A}_H)$ is exponentially small in $d$, with a complementary energy-barrier rate when the noise is small. Along the trajectory, a shape-free bound $ν_t(\mathcal{A}_H) \le π(\mathcal{A}_H)(1 + \sqrt{χ_0^2/π(\mathcal{A}_H)}\,e^{-mt})$ shows the in-set probability relaxes to (twice) the static value after a burn-in of order $d$, using only the global spectral gap $m$. A worked Ornstein-Uhlenbeck example shows this burn-in is necessary: an angular slice of the equilibrium shell can transiently swell by a factor exponential in $d$, though its equilibrium mass is tiny. To rule this out we introduce a local relaxation rate, defined through the spectral measure of the region's centered indicator rather than a Dirichlet-form Rayleigh quotient. For geometrically isolated regions this rate exceeds the global one, shrinking the burn-in, and with a maximum-principle ceiling it caps the trajectory probability uniformly in time. Strong convexity sets how fast training relaxes, but the shape of the unsafe set decides whether the trajectory bulges through it on the way to equilibrium.

Related papers

• KPZ Superdiffusion of Local Correlators in Diffusive Random Quantum Circuits [0.0]
  We find Kardar-Parisi-Zhang scaling in the wandering statistics of the normalized spatial distribution $p(x,t)propto |G(x,t)|2$.<n>In particular, its center $langle x(t)rangleequivsum_x x, p(x,t)$ wanders on a length-scale $mathcalO(t2/3)$, while sample-to-sample fluctuations of $-logsum_x |G(x,t)|2$ scale as
  arXiv  Detail & Related papers  (2026-08-06T18:00:02Z)
• The Information Content of Krylov Observables: A Machine Learning Approach [0.0]
  We quantify the spread complexity $mathcalC(t)$, the discrete Wigner negativity $N(t)$, and the normalized negativity $(t)=N(t)/|S(t)|$, with $S(t)$ the survival amplitude, recently proposed as a second-moment infall probe (arXiv:2607.04065).<n>Small residual networks (16-32 neurons) and boosted trees are trained on half of $sim 57,000$ labeled evolutions spanning the GUE, GOE and Poisson ensembles, the integrable $SL
  arXiv  Detail & Related papers  (2026-07-19T17:17:21Z)
• A Temporal Spatial Minimax Rate for Smoothly-Varying Distributions in Wasserstein Space [0.0]
  We study the minimax rate of estimating a future value $_t_n+h$ of a curve $tmapsto_t$ in the $2$-Wasserstein space.<n>Our central result is a unified temporal-spatial minimax lower bound: over regular, locally transport-rich subclasses.
  arXiv  Detail & Related papers  (2026-06-05T14:43:10Z)
• Local LMO: Constrained Gradient Optimization via a Local Linear Minimization Oracle [51.714334316332476]
  Local L is a new projection-free type for constrained optimization.<n>Local LMO is perhaps best viewed as an oracle of Gradient Descent (GD)
  arXiv  Detail & Related papers  (2026-05-09T10:03:24Z)
• When Does $\ell_2$-Boosting Overfit Benignly? High-Dimensional Risk Asymptotics and the $\ell_1$ Implicit Bias [15.113649527486276]
  We show that benign overfitting fails at the linear rate.<n>We show that while this localization mechanism should persist in the presence of signals, the exact signal-noise decomposition remains an open problem.
  arXiv  Detail & Related papers  (2026-05-07T14:14:09Z)
• On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature [1.6773271875801752]
  Gradient Descent (SGD) introduces anisotropic noise that is correlated with the local curvature of the loss landscape, thereby biasing optimization toward flat minima.<n>We show that this assumption holds only under restrictive conditions that are typically violated in deep neural networks.<n>Experiments across datasets, architectures, and loss functions validate these bounds, providing a unified characterization of the noise-curvature relationship in deep learning.
  arXiv  Detail & Related papers  (2026-02-05T12:35:13Z)
• Anomalous energy correlations and spectral form factor in the nonergodic phase of the $β$-ensemble [0.0]
  We study the dynamical properties of the $beta$-ensemble where the critical energy controls the characteristic timescales.<n>We show that the energy correlations are absent within a temporal window $t_mathrmR t t_mathrmH$, which we term as the correlation void.
  arXiv  Detail & Related papers  (2025-06-18T09:05:22Z)
• A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing [68.80803866919123]
  Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $mathbfx*$ rather than for all $mathbfx*$ simultaneously. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy.
  arXiv  Detail & Related papers  (2023-09-25T17:54:19Z)
• Statistical Learning under Heterogeneous Distribution Shift [71.8393170225794]
  Ground-truth predictor is additive $mathbbE[mathbfz mid mathbfx,mathbfy] = f_star(mathbfx) +g_star(mathbfy)$.
  arXiv  Detail & Related papers  (2023-02-27T16:34:21Z)
• Random quantum circuits transform local noise into global white noise [118.18170052022323]
  We study the distribution over measurement outcomes of noisy random quantum circuits in the low-fidelity regime. For local noise that is sufficiently weak and unital, correlations (measured by the linear cross-entropy benchmark) between the output distribution $p_textnoisy$ of a generic noisy circuit instance shrink exponentially. If the noise is incoherent, the output distribution approaches the uniform distribution $p_textunif$ at precisely the same rate.
  arXiv  Detail & Related papers  (2021-11-29T19:26:28Z)
• Agnostic Learning of Halfspaces with Gradient Descent via Soft Margins [92.7662890047311]
  We show that gradient descent finds halfspaces with classification error $tilde O(mathsfOPT1/2) + varepsilon$ in $mathrmpoly(d,1/varepsilon)$ time and sample complexity.
  arXiv  Detail & Related papers  (2020-10-01T16:48:33Z)
• Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction [63.41789556777387]
  Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a Markov decision process (MDP) We show that the number of samples needed to yield an entrywise $varepsilon$-accurate estimate of the Q-function is at most on the order of $frac1mu_min (1-gamma)5varepsilon2+ fract_mixmu_min (1-gamma)$ up to some logarithmic factor.
  arXiv  Detail & Related papers  (2020-06-04T17:51:00Z)
• Agnostic Learning of a Single Neuron with Gradient Descent [92.7662890047311]
  We consider the problem of learning the best-fitting single neuron as measured by the expected square loss. For the ReLU activation, our population risk guarantee is $O(mathsfOPT1/2)+epsilon$. For the ReLU activation, our population risk guarantee is $O(mathsfOPT1/2)+epsilon$.
  arXiv  Detail & Related papers  (2020-05-29T07:20:35Z)
• Data-driven Efficient Solvers for Langevin Dynamics on Manifold in High Dimensions [12.005576001523515]
  We study the Langevin dynamics of a physical system with manifold structure $mathcalMsubsetmathbbRp$ We leverage the corresponding Fokker-Planck equation on the manifold $mathcalN$ in terms of the reaction coordinates $mathsfy$. We propose an implementable, unconditionally stable, data-driven finite volume scheme for this Fokker-Planck equation.
  arXiv  Detail & Related papers  (2020-05-22T16:55:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.