Deterministic Gibbs Sampling via Ordinary Differential Equations

• URL: http://arxiv.org/abs/2106.10188v1
• Date: Fri, 18 Jun 2021 15:36:09 GMT
• Title: Deterministic Gibbs Sampling via Ordinary Differential Equations
• Authors: Kirill Neklyudov, Roberto Bondesan, Max Welling
• Abstract summary: This paper presents a general construction of deterministic measure-preserving dynamics using autonomous ODEs and tools from differential geometry. We show how Hybrid Monte Carlo and other deterministic samplers follow as special cases of our theory.
• Score: 77.42706423573573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deterministic dynamics is an essential part of many MCMC algorithms, e.g. Hybrid Monte Carlo or samplers utilizing normalizing flows. This paper presents a general construction of deterministic measure-preserving dynamics using autonomous ODEs and tools from differential geometry. We show how Hybrid Monte Carlo and other deterministic samplers follow as special cases of our theory. We then demonstrate the utility of our approach by constructing a continuous non-sequential version of Gibbs sampling in terms of an ODE flow and extending it to discrete state spaces. We find that our deterministic samplers are more sample efficient than stochastic counterparts, even if the latter generate independent samples.

Related papers

• Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis [56.442307356162864]
  We study the theoretical aspects of score-based discrete diffusion models under the Continuous Time Markov Chain (CTMC) framework. We introduce a discrete-time sampling algorithm in the general state space $[S]d$ that utilizes score estimators at predefined time points. Our convergence analysis employs a Girsanov-based method and establishes key properties of the discrete score function.
  arXiv  Detail & Related papers  (2024-10-03T09:07:13Z)
• Stochastic Sampling from Deterministic Flow Models [8.849981177332594]
  We present a method to turn flow models into a family of differential equations (SDEs) that have the same marginal distributions. We empirically demonstrate advantages of our method on a toy Gaussian setup and on the large scale ImageNet generation task.
  arXiv  Detail & Related papers  (2024-10-03T05:18:28Z)
• On the Trajectory Regularity of ODE-based Diffusion Sampling [79.17334230868693]
  Diffusion-based generative models use differential equations to establish a smooth connection between a complex data distribution and a tractable prior distribution. In this paper, we identify several intriguing trajectory properties in the ODE-based sampling process of diffusion models.
  arXiv  Detail & Related papers  (2024-05-18T15:59:41Z)
• A Geometric Perspective on Diffusion Models [57.27857591493788]
  We inspect the ODE-based sampling of a popular variance-exploding SDE. We establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm.
  arXiv  Detail & Related papers  (2023-05-31T15:33:16Z)
• Markov Chain Monte Carlo for Continuous-Time Switching Dynamical Systems [26.744964200606784]
  We propose a novel inference algorithm utilizing a Markov Chain Monte Carlo approach. The presented Gibbs sampler allows to efficiently obtain samples from the exact continuous-time posterior processes.
  arXiv  Detail & Related papers  (2022-05-18T09:03:00Z)
• Continual Repeated Annealed Flow Transport Monte Carlo [93.98285297760671]
  We propose Continual Repeated Annealed Flow Transport Monte Carlo (CRAFT) It combines a sequential Monte Carlo sampler with variational inference using normalizing flows. We show that CRAFT can achieve impressively accurate results on a lattice field example.
  arXiv  Detail & Related papers  (2022-01-31T10:58:31Z)
• Sampling from high-dimensional, multimodal distributions using automatically tuned, tempered Hamiltonian Monte Carlo [0.0]
  Hamiltonian Monte Carlo (HMC) is widely used for sampling from high-dimensional target distributions with probability density known up to proportionality. Traditional tempering methods, commonly used to address multimodality, can be difficult to tune, particularly in high dimensions. We propose a method that combines a tempering strategy with Hamiltonian Monte Carlo, enabling efficient sampling from high-dimensional, strongly multimodal distributions.
  arXiv  Detail & Related papers  (2021-11-12T18:48:36Z)
• Direct sampling of projected entangled-pair states [0.0]
  Variational Monte Carlo studies employing projected entangled-pair states (PEPS) have recently shown that they can provide answers on long-standing questions. We propose a sampling algorithm that generates independent samples from a PEPS, bypassing all problems related to finite autocorrelation times.
  arXiv  Detail & Related papers  (2021-09-15T15:09:20Z)
• Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMC [83.48593305367523]
  Hybrid Monte Carlo is a powerful Markov Chain Monte Carlo method for sampling from complex continuous distributions. We introduce a new approach based on augmenting Monte Carlo methods with SurVAE Flows to sample from discrete distributions. We demonstrate the efficacy of our algorithm on a range of examples from statistics, computational physics and machine learning, and observe improvements compared to alternative algorithms.
  arXiv  Detail & Related papers  (2021-02-04T02:21:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.