Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling

• URL: http://arxiv.org/abs/2608.25052v1
• Date: Tue, 25 Aug 2026 18:42:23 GMT
• Title: Improved Analysis for Hessian-free High-resolution Monte Carlo Sampling
• Abstract summary: Hessian-free high-resolution (HFHR) dynamics augments underdamped Langevin dynamics (ULD)<n>We establish an explicit quantitative contraction rate for HFHR dynamics under a position Poincaré inequality.<n>An adapted time-augmented Poincaré inequality yields an explicit rate that improves upon the contraction rate of the Langevin dynamics.
• Score: 7.336637417508747
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hessian-free high-resolution (HFHR) dynamics augments underdamped Langevin dynamics (ULD) with reversible position diffusion for sampling problems that arise in machine learning. We establish an explicit quantitative contraction rate for HFHR dynamics under a position Poincaré inequality, weighted Hessian and Laplacian bounds, and a compact Sobolev embedding, where the potential function is not necessarily convex. An adapted time-augmented Poincaré inequality yields an explicit rate that improves upon the contraction rate of the underdamped Langevin dynamics. We also give a weak-solution construction and a self-contained spectral proof of the divergence lemma underlying the argument. For HFHR Monte Carlo (HFHRMC) algorithm, which is based on a discretization scheme of HFHR dynamics, we use a path-space Girsanov argument to obtain a non-asymptotic convergence bound and an explicit iteration complexity in total variation distance. The bounds hold for every $α\geq0$ and $γ>0$ and remain regular at the ULD endpoint. Optimizing the iteration complexity bound yields a positive, accuracy-dependent position-diffusion parameter at finite accuracy, while its leading high-accuracy order coincides with that of the optimized ULD endpoint. Our iteration complexity bound improves upon the existing work on HFHR algorithms. Numerical experiments including Bayesian learning problems on real data are provided to illustrate the effect of positive $α$ and its benefit.

Related papers

• Error estimates for tamed Euler and Randomized Euler schemes for SDEs with locally Lipschitz drift with applications to non-logconcave sampling and optimization [0.0]
  We tame the numerical discretization of differential equations with locally Lipschitz, super-optimally growing drift, and the resulting implications for sampling from non-log-concave distributions satisfying a Sobolev inequality.<n>We establish, for the first time, a non-asymotic guarantee in total variation for a tamed randomized Langevin scheme under superlinear growth.
  arXiv  Detail & Related papers  (2026-05-24T08:35:24Z)
• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Sampling non-log-concave densities via Hessian-free high-resolution dynamics [8.014749318189851]
  We study the problem of sampling from a target distribution $(q)prop e-U(q) of the Hessian-free-resolution (HFHR) process.<n>We show that, under an additional assumption that $blana U$ linear growth has at infinity, the contraction for HFHR dynamics is strictly better than that KLD is.
  arXiv  Detail & Related papers  (2026-01-06T05:38:09Z)
• Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations [57.179679246370114]
  We identify the distribution of random perturbations that minimizes the estimator's variance as the perturbation stepsize tends to zero.<n>Our findings reveal that such desired perturbations can align directionally with the true gradient, instead of maintaining a fixed length.
  arXiv  Detail & Related papers  (2025-10-22T19:06:39Z)
• Flatness-Aware Stochastic Gradient Langevin Dynamics [8.884140046635268]
  Generalization in deep learning algorithms converge to the pursuit of flat minima in the loss landscape.<n>We show that Gradient Langevin Dynamics (SGLD) offers no bias to establish such low-trace regularized objective.
  arXiv  Detail & Related papers  (2025-10-02T16:24:46Z)
• Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization [5.582101184758528]
  We prove strong convergence of reg-SGD to the minimum-norm solution of the original problem without additional boundedness assumptions.<n>Our analysis reveals how vanishing Tikhonov regularization controls the flow of SGD and yields stable learning dynamics.
  arXiv  Detail & Related papers  (2025-05-16T16:53:49Z)
• Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems [78.96969465641024]
  We extend mean-field Langevin dynamics to minimax optimization over probability distributions for the first time with symmetric and provably convergent updates. We also study time and particle discretization regimes and prove a new uniform-in-time propagation of chaos result.
  arXiv  Detail & Related papers  (2023-12-02T13:01:29Z)
• Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction [49.66486092259376]
  The mean-field Langevin dynamics (MFLD) is a nonlinear generalization of the Langevin dynamics that incorporates a distribution-dependent drift. Recent works have shown that MFLD globally minimizes an entropy-regularized convex functional in the space of measures. We provide a framework to prove a uniform-in-time propagation of chaos for MFLD that takes into account the errors due to finite-particle approximation, time-discretization, and gradient approximation.
  arXiv  Detail & Related papers  (2023-06-12T16:28:11Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Hessian-Free High-Resolution Nesterov Acceleration for Sampling [55.498092486970364]
  Nesterov's Accelerated Gradient (NAG) for optimization has better performance than its continuous time limit (noiseless kinetic Langevin) when a finite step-size is employed. This work explores the sampling counterpart of this phenonemon and proposes a diffusion process, whose discretizations can yield accelerated gradient-based MCMC methods.
  arXiv  Detail & Related papers  (2020-06-16T15:07:37Z)
• Non-Convex Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics [27.097121544378528]
  Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non- objective gradient. NSGLD is based on discretization of the non-reversible diffusion.
  arXiv  Detail & Related papers  (2020-04-06T17:11:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.