Policy Gradients for Optimal Parallel Tempering MCMC

• URL: http://arxiv.org/abs/2409.01574v1
• Date: Tue, 3 Sep 2024 03:12:45 GMT
• Title: Policy Gradients for Optimal Parallel Tempering MCMC
• Authors: Daniel Zhao, Natesh S. Pillai,
• Abstract summary: Parallel tempering is a meta-algorithm for Markov Chain Monte Carlo that uses multiple chains to sample from tempered versions of the target distribution. We present an adaptive temperature selection algorithm that dynamically adjusts temperatures during sampling using a policy gradient approach.
• Score: 0.276240219662896
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Parallel tempering is meta-algorithm for Markov Chain Monte Carlo that uses multiple chains to sample from tempered versions of the target distribution, enhancing mixing in multi-modal distributions that are challenging for traditional methods. The effectiveness of parallel tempering is heavily influenced by the selection of chain temperatures. Here, we present an adaptive temperature selection algorithm that dynamically adjusts temperatures during sampling using a policy gradient approach. Experiments demonstrate that our method can achieve lower integrated autocorrelation times compared to traditional geometrically spaced temperatures and uniform acceptance rate schemes on benchmark distributions.

Related papers

• Optimization of Discrete Parameters Using the Adaptive Gradient Method and Directed Evolution [49.1574468325115]
  The search for an optimal solution is carried out by a population of individuals. Unadapted individuals die, and optimal ones interbreed, the result directed evolutionary dynamics.
  arXiv  Detail & Related papers  (2024-01-12T15:45:56Z)
• A connection between Tempering and Entropic Mirror Descent [8.775514582692795]
  We establish that tempering SMC corresponds to entropic mirror descent applied to the reverse Kullback-Leibler divergence. We derive adaptive tempering rules that improve over other alternative benchmarks in the literature.
  arXiv  Detail & Related papers  (2023-10-18T12:06:47Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Differentiating Metropolis-Hastings to Optimize Intractable Densities [51.16801956665228]
  We develop an algorithm for automatic differentiation of Metropolis-Hastings samplers. We apply gradient-based optimization to objectives expressed as expectations over intractable target densities.
  arXiv  Detail & Related papers  (2023-06-13T17:56:02Z)
• Sampling with Mollified Interaction Energy Descent [57.00583139477843]
  We present a new optimization-based method for sampling called mollified interaction energy descent (MIED) MIED minimizes a new class of energies on probability measures called mollified interaction energies (MIEs) We show experimentally that for unconstrained sampling problems our algorithm performs on par with existing particle-based algorithms like SVGD.
  arXiv  Detail & Related papers  (2022-10-24T16:54:18Z)
• AdaAnn: Adaptive Annealing Scheduler for Probability Density Approximation [3.1370892256881255]
  Annealing can be used to facilitate Approximating probability distributions over regions of high geometrical complexity. We introduce AdaAnn, an adaptive scheduler that automatically adjusts the temperature increments based on the expected change in the Kullback-Leibler divergence. AdaAnn is easy to implement and can be integrated into existing sampling approaches such as normalizing flows for variational inference and Markov chain Monte Carlo.
  arXiv  Detail & Related papers  (2022-02-01T22:26:18Z)
• Renormalization group for open quantum systems using environment temperature as flow parameter [0.0]
  We present the $T$-flow renormalization group method, which computes the memory kernel for the density-operator evolution of an open quantum system. We benchmark in the stationary limit, readily accessible in real-time for voltages on the order of the coupling or larger. We analytically show that the short-time dynamics of both local and non-local observables follow a universal temperature-independent behaviour.
  arXiv  Detail & Related papers  (2021-11-14T11:52:27Z)
• Sampling from high-dimensional, multimodal distributions using automatically tuned, tempered Hamiltonian Monte Carlo [0.0]
  Hamiltonian Monte Carlo (HMC) is widely used for sampling from high-dimensional target distributions with probability density known up to proportionality. Traditional tempering methods, commonly used to address multimodality, can be difficult to tune, particularly in high dimensions. We propose a method that combines a tempering strategy with Hamiltonian Monte Carlo, enabling efficient sampling from high-dimensional, strongly multimodal distributions.
  arXiv  Detail & Related papers  (2021-11-12T18:48:36Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)
• Adaptive Correlated Monte Carlo for Contextual Categorical Sequence Generation [77.7420231319632]
  We adapt contextual generation of categorical sequences to a policy gradient estimator, which evaluates a set of correlated Monte Carlo (MC) rollouts for variance control. We also demonstrate the use of correlated MC rollouts for binary-tree softmax models, which reduce the high generation cost in large vocabulary scenarios.
  arXiv  Detail & Related papers  (2019-12-31T03:01:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.