Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics

• URL: http://arxiv.org/abs/2410.09697v1
• Date: Sun, 13 Oct 2024 02:24:31 GMT
• Title: Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics
• Authors: Omar Chehab, Anna Korba, Austin Stromme, Adrien Vacher,
• Abstract summary: Geometric tempering is a popular approach to sampling from challenging multi-modal probability distributions. In this paper, we theoretically investigate the soundness of this approach when the sampling algorithm is Langevin dynamics. Our results indicate that geometric tempering may not help, and can even be harmful for convergence.
• Score: 8.683011785637824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geometric tempering is a popular approach to sampling from challenging multi-modal probability distributions by instead sampling from a sequence of distributions which interpolate, using the geometric mean, between an easier proposal distribution and the target distribution. In this paper, we theoretically investigate the soundness of this approach when the sampling algorithm is Langevin dynamics, proving both upper and lower bounds. Our upper bounds are the first analysis in the literature under functional inequalities. They assert the convergence of tempered Langevin in continuous and discrete-time, and their minimization leads to closed-form optimal tempering schedules for some pairs of proposal and target distributions. Our lower bounds demonstrate a simple case where the geometric tempering takes exponential time, and further reveal that the geometric tempering can suffer from poor functional inequalities and slow convergence, even when the target distribution is well-conditioned. Overall, our results indicate that geometric tempering may not help, and can even be harmful for convergence.

Related papers

• Constrained Sampling with Primal-Dual Langevin Monte Carlo [15.634831573546041]
  This work considers the problem of sampling from a probability distribution known up to a normalization constant. It satisfies a set of statistical constraints specified by the expected values of general nonlinear functions. We put forward a discrete-time primal-dual Langevin Monte Carlo algorithm (PD-LMC) that simultaneously constrains the target distribution and samples from it.
  arXiv  Detail & Related papers  (2024-11-01T13:26:13Z)
• An Improved Analysis of Langevin Algorithms with Prior Diffusion for Non-Log-Concave Sampling [27.882407333690267]
  We show that modified Langevin algorithm with prior diffusion can converge dimension independently for strongly log-concave target distributions. We also prove that the modified Langevin algorithm can also obtain the dimension-independent convergence of KL divergence with different step size schedules.
  arXiv  Detail & Related papers  (2024-03-10T11:50:34Z)
• Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems [78.96969465641024]
  We extend mean-field Langevin dynamics to minimax optimization over probability distributions for the first time with symmetric and provably convergent updates. We also study time and particle discretization regimes and prove a new uniform-in-time propagation of chaos result.
  arXiv  Detail & Related papers  (2023-12-02T13:01:29Z)
• Taming under isoperimetry [0.0]
  In this article we propose a Langevin-based scheme calledmathbfsTULA$ to sample from distributions with growing log. We derive non-asymientKL and consequently consequently satisfy a Log-Sobolev inequality.
  arXiv  Detail & Related papers  (2023-11-15T14:44:16Z)
• Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds [77.4346324549323]
  We show that a step size agnostic to the curvature of the manifold achieves a curvature-independent and linear last-iterate convergence rate. To the best of our knowledge, the possibility of curvature-independent rates and/or last-iterate convergence has not been considered before.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Convergence of the Riemannian Langevin Algorithm [10.279748604797911]
  We study the problem of sampling from a distribution with density $nu$ with respect to the natural measure on a manifold with metric $g$. A special case of our approach is sampling isoperimetric densities restricted to polytopes defined by the logarithmic barrier.
  arXiv  Detail & Related papers  (2022-04-22T16:56:00Z)
• Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems [98.34292831923335]
  Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm. We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
  arXiv  Detail & Related papers  (2021-12-29T18:46:52Z)
• Unadjusted Langevin algorithm for sampling a mixture of weakly smooth potentials [0.0]
  We prove convergence guarantees under Poincar'e inequality or non-strongly convex outside the ball. We also provide convergence in $L_beta$-Wasserstein metric for the smoothing potential.
  arXiv  Detail & Related papers  (2021-12-17T04:10:09Z)
• Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond [63.59034509960994]
  We study shuffling-based variants: minibatch and local Random Reshuffling, which draw gradients without replacement. For smooth functions satisfying the Polyak-Lojasiewicz condition, we obtain convergence bounds which show that these shuffling-based variants converge faster than their with-replacement counterparts. We propose an algorithmic modification called synchronized shuffling that leads to convergence rates faster than our lower bounds in near-homogeneous settings.
  arXiv  Detail & Related papers  (2021-10-20T02:25:25Z)
• Mean-Square Analysis with An Application to Optimal Dimension Dependence of Langevin Monte Carlo [60.785586069299356]
  This work provides a general framework for the non-asymotic analysis of sampling error in 2-Wasserstein distance. Our theoretical analysis is further validated by numerical experiments.
  arXiv  Detail & Related papers  (2021-09-08T18:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.