Second Order Ensemble Langevin Method for Sampling and Inverse Problems

• URL: http://arxiv.org/abs/2208.04506v1
• Date: Tue, 9 Aug 2022 02:17:22 GMT
• Title: Second Order Ensemble Langevin Method for Sampling and Inverse Problems
• Authors: Ziming Liu, Andrew M. Stuart, Yixuan Wang
• Abstract summary: We propose a sampling method based on an ensemble approximation of Langevin dynamics. Numerical results demonstrate its potential as the basis for a numerical sampler in inverse problems.
• Score: 10.406582941856099
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a sampling method based on an ensemble approximation of second order Langevin dynamics. The log target density is appended with a quadratic term in an auxiliary momentum variable and damped-driven Hamiltonian dynamics introduced; the resulting stochastic differential equation is invariant to the Gibbs measure, with marginal on the position coordinates given by the target. A preconditioner based on covariance under the law of the dynamics does not change this invariance property, and is introduced to accelerate convergence to the Gibbs measure. The resulting mean-field dynamics may be approximated by an ensemble method; this results in a gradient-free and affine-invariant stochastic dynamical system. Numerical results demonstrate its potential as the basis for a numerical sampler in Bayesian inverse problems.

Related papers

• Analytical Approximation of the ELBO Gradient in the Context of the Clutter Problem [0.0]
  We propose an analytical solution for approximating the gradient of the Evidence Lower Bound (ELBO) in variational inference problems. The proposed method demonstrates good accuracy and rate of convergence together with linear computational complexity.
  arXiv  Detail & Related papers  (2024-04-16T13:19:46Z)
• Proximal Algorithms for Accelerated Langevin Dynamics [57.08271964961975]
  We develop a novel class of MCMC algorithms based on a stochastized Nesterov scheme. We show superior performance of the proposed method over typical Langevin samplers for different models in statistics and image processing.
  arXiv  Detail & Related papers  (2023-11-24T19:56:01Z)
• Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals [3.4240632942024685]
  We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score based MCMC method that is deterministic, resulting in a deterministic evolution for particles.
  arXiv  Detail & Related papers  (2023-08-28T23:51:33Z)
• Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations [0.0]
  Variational mixtures with full-covariance structures suffer from a quadratic growth due to variational parameters with the number of parameters. We propose a method for constructing an initial Gaussian model approximation that can be used to warm-start variational inference.
  arXiv  Detail & Related papers  (2023-07-12T19:30:04Z)
• Stochastic parameter optimization analysis of dynamical quantum critical phenomena in long-range transverse-field Ising chain [0.0]
  We explore the quantum phase transition of the one-dimensional long-range transverse-field Ising model. In our simulations, the simulator automatically determines the parameters to sample from, even without prior knowledge of the critical point and universality class. We successfully obtained numerical evidence supporting $sigma = 7/4$ as the universality boundary between the latter two.
  arXiv  Detail & Related papers  (2023-05-23T14:46:16Z)
• Improved Convergence Rate of Stochastic Gradient Langevin Dynamics with Variance Reduction and its Application to Optimization [50.83356836818667]
  gradient Langevin Dynamics is one of the most fundamental algorithms to solve non-eps optimization problems. In this paper, we show two variants of this kind, namely the Variance Reduced Langevin Dynamics and the Recursive Gradient Langevin Dynamics.
  arXiv  Detail & Related papers  (2022-03-30T11:39:00Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Determination of the critical exponents in dissipative phase transitions: Coherent anomaly approach [51.819912248960804]
  We propose a generalization of the coherent anomaly method to extract the critical exponents of a phase transition occurring in the steady-state of an open quantum many-body system.
  arXiv  Detail & Related papers  (2021-03-12T13:16:18Z)
• Bernstein-Greene-Kruskal approach for the quantum Vlasov equation [91.3755431537592]
  The one-dimensional stationary quantum Vlasov equation is analyzed using the energy as one of the dynamical variables. In the semiclassical case where quantum tunneling effects are small, an infinite series solution is developed.
  arXiv  Detail & Related papers  (2021-02-18T20:55:04Z)
• The Connection between Discrete- and Continuous-Time Descriptions of Gaussian Continuous Processes [60.35125735474386]
  We show that discretizations yielding consistent estimators have the property of invariance under coarse-graining' This result explains why combining differencing schemes for derivatives reconstruction and local-in-time inference approaches does not work for time series analysis of second or higher order differential equations.
  arXiv  Detail & Related papers  (2021-01-16T17:11:02Z)
• Generalized Gumbel-Softmax Gradient Estimator for Various Discrete Random Variables [16.643346012854156]
  Esting the gradients of nodes is one of the crucial research questions in the deep generative modeling community. This paper proposes a general version of the Gumbel-Softmax estimator with continuous relaxation.
  arXiv  Detail & Related papers  (2020-03-04T01:13:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.