Flow Perturbation to Accelerate Unbiased Sampling of Boltzmann distribution

• URL: http://arxiv.org/abs/2407.10666v2
• Date: Sat, 27 Jul 2024 04:52:29 GMT
• Title: Flow Perturbation to Accelerate Unbiased Sampling of Boltzmann distribution
• Authors: Xin Peng, Ang Gao,
• Abstract summary: Flow-based generative models have been employed for sampling the Boltzmann distribution, but their application is hindered by the computational cost of obtaining the Jacobian of the flow. We introduce the flow perturbation method, which incorporates optimized perturbations into the flow. By reweighting trajectories generated by the perturbed flow, our method achieves unbiased sampling of the Boltzmann distribution with orders of magnitude speedup.
• Score: 2.103187931015573
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Flow-based generative models have been employed for sampling the Boltzmann distribution, but their application to high-dimensional systems is hindered by the significant computational cost of obtaining the Jacobian of the flow. To overcome this challenge, we introduce the flow perturbation method, which incorporates optimized stochastic perturbations into the flow. By reweighting trajectories generated by the perturbed flow, our method achieves unbiased sampling of the Boltzmann distribution with orders of magnitude speedup compared to both brute force Jacobian calculations and the Hutchinson estimator. Notably, it accurately sampled the Chignolin protein with all atomic Cartesian coordinates explicitly represented, which, to our best knowledge, is the largest molecule ever Boltzmann sampled in such detail using generative models.

Related papers

• Verlet Flows: Exact-Likelihood Integrators for Flow-Based Generative Models [4.9425328004453375]
  We present Verlet flows, a class of CNFs on an augmented state-space inspired by symplectic from Hamiltonian dynamics. Verlet flows provide exact-likelihood generative models which generalize coupled flow architectures from a non-continuous setting while imposing minimal expressivity constraints. On experiments over toy densities, we demonstrate that the variance of the commonly used Hutchinson trace estimator is unsuitable for importance sampling, whereas Verlet flows perform comparably to full autograd trace computations while being significantly faster.
  arXiv  Detail & Related papers  (2024-05-05T03:47:56Z)
• Gaussian Interpolation Flows [11.340847429991525]
  This work investigates the well-posedness of simulation-free continuous normalizing flows built on Gaussian denoising. We establish the Lipschitz regularity of the flow velocity field, the existence and uniqueness of the flow, and the continuity of the flow map. We also study the stability of these flows in source distributions and perturbations of the velocity field, using the quadratic Wasserstein distance as a metric.
  arXiv  Detail & Related papers  (2023-11-20T00:59:20Z)
• Analysis of learning a flow-based generative model from limited sample complexity [39.771578460963774]
  We study the problem of training a flow-based generative model, parametrized by a two-layer autoencoder, to sample from a high-dimensional Gaussian mixture.
  arXiv  Detail & Related papers  (2023-10-05T14:53:40Z)
• Noise-Free Sampling Algorithms via Regularized Wasserstein Proximals [3.4240632942024685]
  We consider the problem of sampling from a distribution governed by a potential function. This work proposes an explicit score based MCMC method that is deterministic, resulting in a deterministic evolution for particles.
  arXiv  Detail & Related papers  (2023-08-28T23:51:33Z)
• ManiFlow: Implicitly Representing Manifolds with Normalizing Flows [145.9820993054072]
  Normalizing Flows (NFs) are flexible explicit generative models that have been shown to accurately model complex real-world data distributions. We propose an optimization objective that recovers the most likely point on the manifold given a sample from the perturbed distribution. Finally, we focus on 3D point clouds for which we utilize the explicit nature of NFs, i.e. surface normals extracted from the gradient of the log-likelihood and the log-likelihood itself.
  arXiv  Detail & Related papers  (2022-08-18T16:07:59Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Flow-based sampling in the lattice Schwinger model at criticality [54.48885403692739]
  Flow-based algorithms may provide efficient sampling of field distributions for lattice field theory applications. We provide a numerical demonstration of robust flow-based sampling in the Schwinger model at the critical value of the fermion mass.
  arXiv  Detail & Related papers  (2022-02-23T19:00:00Z)
• Resampling Base Distributions of Normalizing Flows [0.0]
  We introduce a base distribution for normalizing flows based on learned rejection sampling. We develop suitable learning algorithms using both maximizing the log-likelihood and the optimization of the reverse Kullback-Leibler divergence.
  arXiv  Detail & Related papers  (2021-10-29T14:44:44Z)
• Instance-Optimal Compressed Sensing via Posterior Sampling [101.43899352984774]
  We show for Gaussian measurements and emphany prior distribution on the signal, that the posterior sampling estimator achieves near-optimal recovery guarantees. We implement the posterior sampling estimator for deep generative priors using Langevin dynamics, and empirically find that it produces accurate estimates with more diversity than MAP.
  arXiv  Detail & Related papers  (2021-06-21T22:51:56Z)
• Gaussianization Flows [113.79542218282282]
  We propose a new type of normalizing flow model that enables both efficient iteration of likelihoods and efficient inversion for sample generation. Because of this guaranteed expressivity, they can capture multimodal target distributions without compromising the efficiency of sample generation.
  arXiv  Detail & Related papers  (2020-03-04T08:15:06Z)
• Generative Modeling with Denoising Auto-Encoders and Langevin Sampling [88.83704353627554]
  We show that both DAE and DSM provide estimates of the score of the smoothed population density. We then apply our results to the homotopy method of arXiv:1907.05600 and provide theoretical justification for its empirical success.
  arXiv  Detail & Related papers  (2020-01-31T23:50:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.