Sampling from multimodal distributions with warm starts: Non-asymptotic bounds for the Reweighted Annealed Leap-Point Sampler

• URL: http://arxiv.org/abs/2512.17977v1
• Date: Fri, 19 Dec 2025 12:11:16 GMT
• Title: Sampling from multimodal distributions with warm starts: Non-asymptotic bounds for the Reweighted Annealed Leap-Point Sampler
• Authors: Holden Lee, Matheau Santana-Gijzen,
• Abstract summary: We introduce Reweighted ALPS (Re-ALPS), a modified approximation of the Annealed Leap-Point Sampler (ALPS)<n>We define distributions tilted towards a mixture centered at the warm start points, and at the coldest level, use teleportation between warm start points to enable efficient mixing across modes.<n>In contrast to ALPS, our method does not require Hessian information at the modes, but instead estimates component partition functions via Monte Carlo.
• Score: 10.161956880665734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sampling from multimodal distributions is a central challenge in Bayesian inference and machine learning. In light of hardness results for sampling -- classical MCMC methods, even with tempering, can suffer from exponential mixing times -- a natural question is how to leverage additional information, such as a warm start point for each mode, to enable faster mixing across modes. To address this, we introduce Reweighted ALPS (Re-ALPS), a modified version of the Annealed Leap-Point Sampler (ALPS) that dispenses with the Gaussian approximation assumption. We prove the first polynomial-time bound that works in a general setting, under a natural assumption that each component contains significant mass relative to the others when tilted towards the corresponding warm start point. Similarly to ALPS, we define distributions tilted towards a mixture centered at the warm start points, and at the coldest level, use teleportation between warm start points to enable efficient mixing across modes. In contrast to ALPS, our method does not require Hessian information at the modes, but instead estimates component partition functions via Monte Carlo. This additional estimation step is crucial in allowing the algorithm to handle target distributions with more complex geometries besides approximate Gaussian. For the proof, we show convergence results for Markov processes when only part of the stationary distribution is well-mixing and estimation for partition functions for individual components of a mixture. We numerically evaluate our algorithm's mixing performance compared to ALPS on a mixture of heavy-tailed distributions.

Related papers

• Mixtures Closest to a Given Measure: A Semidefinite Programming Approach [1.7969777786551424]
  We study the problem of approximating a target measure, available only through finitely many of its moments.<n>Unlike many existing approaches, the parameter set is not assumed to be finite.<n>We present an application to clustering, where our framework serves as a stand-alone method or as a preprocessing step.
  arXiv  Detail & Related papers  (2025-09-26T19:51:21Z)
• Bouncy particle sampler with infinite exchanging parallel tempering [0.0]
  We employ the variational Bayesian inference or sampling method to approximate posterior distributions.<n>A bouncy particle sampler (BPS) has been proposed, which combines uniform linear motion and reflection to perform sampling.<n>We performed numerical simulations and demonstrated its effectiveness for multimodal distribution.
  arXiv  Detail & Related papers  (2025-09-02T06:37:57Z)
• Feynman-Kac Correctors in Diffusion: Annealing, Guidance, and Product of Experts [64.34482582690927]
  We provide an efficient and principled method for sampling from a sequence of annealed, geometric-averaged, or product distributions derived from pretrained score-based models.<n>We propose Sequential Monte Carlo (SMC) resampling algorithms that leverage inference-time scaling to improve sampling quality.
  arXiv  Detail & Related papers  (2025-03-04T17:46:51Z)
• Sampling from multi-modal distributions with polynomial query complexity in fixed dimension via reverse diffusion [16.463220658992064]
  We provide the first sampling algorithm for a broad class of distributions.<n>Our algorithm simulates a time-reversed diffusion process.<n>It avoids metastability, requires no prior knowledge of the mode locations, and relaxes the well-known log-smoothness assumption.
  arXiv  Detail & Related papers  (2024-12-31T17:51:39Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Interpolating between sampling and variational inference with infinite stochastic mixtures [6.021787236982659]
  Sampling and Variational Inference (VI) are two large families of methods for approximate inference with complementary strengths. Here, we develop a framework for constructing intermediate algorithms that balance the strengths of both sampling and VI. This work is a first step towards a highly flexible yet simple family of inference methods that combines the complementary strengths of sampling and VI.
  arXiv  Detail & Related papers  (2021-10-18T20:50:06Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)
• Plug-And-Play Learned Gaussian-mixture Approximate Message Passing [71.74028918819046]
  We propose a plug-and-play compressed sensing (CS) recovery algorithm suitable for any i.i.d. source prior. Our algorithm builds upon Borgerding's learned AMP (LAMP), yet significantly improves it by adopting a universal denoising function within the algorithm. Numerical evaluation shows that the L-GM-AMP algorithm achieves state-of-the-art performance without any knowledge of the source prior.
  arXiv  Detail & Related papers  (2020-11-18T16:40:45Z)
• Pathwise Conditioning of Gaussian Processes [72.61885354624604]
  Conventional approaches for simulating Gaussian process posteriors view samples as draws from marginal distributions of process values at finite sets of input locations. This distribution-centric characterization leads to generative strategies that scale cubically in the size of the desired random vector. We show how this pathwise interpretation of conditioning gives rise to a general family of approximations that lend themselves to efficiently sampling Gaussian process posteriors.
  arXiv  Detail & Related papers  (2020-11-08T17:09:37Z)
• Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction [63.41789556777387]
  Asynchronous Q-learning aims to learn the optimal action-value function (or Q-function) of a Markov decision process (MDP) We show that the number of samples needed to yield an entrywise $varepsilon$-accurate estimate of the Q-function is at most on the order of $frac1mu_min (1-gamma)5varepsilon2+ fract_mixmu_min (1-gamma)$ up to some logarithmic factor.
  arXiv  Detail & Related papers  (2020-06-04T17:51:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.