Convergence of Kinetic Langevin Monte Carlo on Lie groups
- URL: http://arxiv.org/abs/2403.12012v2
- Date: Tue, 18 Jun 2024 01:08:24 GMT
- Title: Convergence of Kinetic Langevin Monte Carlo on Lie groups
- Authors: Lingkai Kong, Molei Tao,
- Abstract summary: We propose a Lie-group MCMC sampler, by delicately discretizing the resulting kinetic-Langevin-type sampling dynamics.
This is the first convergence result for kinetic Langevin on curved spaces, and also the first quantitative result that requires no convexity or, at least not explicitly, any common relaxation such as isoperimetry.
- Score: 21.76159063788814
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Explicit, momentum-based dynamics for optimizing functions defined on Lie groups was recently constructed, based on techniques such as variational optimization and left trivialization. We appropriately add tractable noise to the optimization dynamics to turn it into a sampling dynamics, leveraging the advantageous feature that the trivialized momentum variable is Euclidean despite that the potential function lives on a manifold. We then propose a Lie-group MCMC sampler, by delicately discretizing the resulting kinetic-Langevin-type sampling dynamics. The Lie group structure is exactly preserved by this discretization. Exponential convergence with explicit convergence rate for both the continuous dynamics and the discrete sampler are then proved under $W_2$ distance. Only compactness of the Lie group and geodesically $L$-smoothness of the potential function are needed. To the best of our knowledge, this is the first convergence result for kinetic Langevin on curved spaces, and also the first quantitative result that requires no convexity or, at least not explicitly, any common relaxation such as isoperimetry.
Related papers
- von Mises Quasi-Processes for Bayesian Circular Regression [57.88921637944379]
We explore a family of expressive and interpretable distributions over circle-valued random functions.
The resulting probability model has connections with continuous spin models in statistical physics.
For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Markov Chain Monte Carlo sampling.
arXiv Detail & Related papers (2024-06-19T01:57:21Z) - Quantitative Convergences of Lie Group Momentum Optimizers [21.76159063788814]
This article investigates two types of discretization, Lie Heavy-Ball and Lie NAG-SC, which is newly proposed.
Lie Heavy-Ball and Lie NAG-SC are computationally cheaper and easier to implement, thanks to their utilization of group structure.
arXiv Detail & Related papers (2024-05-30T18:01:14Z) - Fisher information dissipation for time inhomogeneous stochastic
differential equations [7.076726009680242]
We provide a Lyapunov convergence analysis for time-inhomogeneous variable coefficient differential equations.
Three typical examples include overdamped, irreversible drift, and underdamped Langevin dynamics.
arXiv Detail & Related papers (2024-02-01T21:49:50Z) - Symmetric Mean-field Langevin Dynamics for Distributional Minimax
Problems [78.96969465641024]
We extend mean-field Langevin dynamics to minimax optimization over probability distributions for the first time with symmetric and provably convergent updates.
We also study time and particle discretization regimes and prove a new uniform-in-time propagation of chaos result.
arXiv Detail & Related papers (2023-12-02T13:01:29Z) - Generating function for projected entangled-pair states [0.1759252234439348]
We extend the generating function approach for tensor network diagrammatic summation.
Taking the form of a one-particle excitation, we show that the excited state can be computed efficiently in the generating function formalism.
We conclude with a discussion on generalizations to multi-particle excitations.
arXiv Detail & Related papers (2023-07-16T15:49:37Z) - Second Order Ensemble Langevin Method for Sampling and Inverse Problems [10.406582941856099]
We propose a sampling method based on an ensemble approximation of Langevin dynamics.
Numerical results demonstrate its potential as the basis for a numerical sampler in inverse problems.
arXiv Detail & Related papers (2022-08-09T02:17:22Z) - Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with
Variance Reduction and its Application to Optimization [50.83356836818667]
gradient Langevin Dynamics is one of the most fundamental algorithms to solve non-eps optimization problems.
In this paper, we show two variants of this kind, namely the Variance Reduced Langevin Dynamics and the Recursive Gradient Langevin Dynamics.
arXiv Detail & Related papers (2022-03-30T11:39:00Z) - Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
convergence rate analysis of the mean field Langevin dynamics is presented.
$p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
arXiv Detail & Related papers (2022-01-25T17:13:56Z) - Sensing Cox Processes via Posterior Sampling and Positive Bases [56.82162768921196]
We study adaptive sensing of point processes, a widely used model from spatial statistics.
We model the intensity function as a sample from a truncated Gaussian process, represented in a specially constructed positive basis.
Our adaptive sensing algorithms use Langevin dynamics and are based on posterior sampling (textscCox-Thompson) and top-two posterior sampling (textscTop2) principles.
arXiv Detail & Related papers (2021-10-21T14:47:06Z) - Multiplicative noise and heavy tails in stochastic optimization [62.993432503309485]
empirical optimization is central to modern machine learning, but its role in its success is still unclear.
We show that it commonly arises in parameters of discrete multiplicative noise due to variance.
A detailed analysis is conducted in which we describe on key factors, including recent step size, and data, all exhibit similar results on state-of-the-art neural network models.
arXiv Detail & Related papers (2020-06-11T09:58:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.