Iterated Energy-based Flow Matching for Sampling from Boltzmann Densities

• URL: http://arxiv.org/abs/2408.16249v1
• Date: Thu, 29 Aug 2024 04:06:34 GMT
• Title: Iterated Energy-based Flow Matching for Sampling from Boltzmann Densities
• Authors: Dongyeop Woo, Sungsoo Ahn,
• Abstract summary: We propose iterated energy-based flow matching (iEFM) to train continuous normalizing flow (CNF) models from unnormalized densities. Our results demonstrate that iEFM outperforms existing methods, showcasing its potential for efficient and scalable probabilistic modeling.
• Score: 11.850515912491657
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we consider the problem of training a generator from evaluations of energy functions or unnormalized densities. This is a fundamental problem in probabilistic inference, which is crucial for scientific applications such as learning the 3D coordinate distribution of a molecule. To solve this problem, we propose iterated energy-based flow matching (iEFM), the first off-policy approach to train continuous normalizing flow (CNF) models from unnormalized densities. We introduce the simulation-free energy-based flow matching objective, which trains the model to predict the Monte Carlo estimation of the marginal vector field constructed from known energy functions. Our framework is general and can be extended to variance-exploding (VE) and optimal transport (OT) conditional probability paths. We evaluate iEFM on a two-dimensional Gaussian mixture model (GMM) and an eight-dimensional four-particle double-well potential (DW-4) energy function. Our results demonstrate that iEFM outperforms existing methods, showcasing its potential for efficient and scalable probabilistic modeling in complex high-dimensional systems.

Related papers

• Zero-point energy of tensor fluctuations on the MPS manifold [0.05524804393257919]
  This work presents a method for studying low-energy physics in highly correlated magnetic systems using the matrix product state (MPS) manifold. We adapt the spin-wave approach, which has been very successful in modeling certain low-entanglement magnetic materials, to systems where the ground state is better represented by an MPS.
  arXiv  Detail & Related papers  (2024-10-29T18:00:02Z)
• Hessian-Informed Flow Matching [4.542719108171107]
  Hessian-Informed Flow Matching is a novel approach that integrates the Hessian of an energy function into conditional flows. This integration allows HI-FM to account for local curvature and anisotropic covariance structures. Empirical evaluations on the MNIST and Lennard-Jones particles datasets demonstrate that HI-FM improves the likelihood of test samples.
  arXiv  Detail & Related papers  (2024-10-15T09:34:52Z)
• Flow matching achieves almost minimax optimal convergence [50.38891696297888]
  Flow matching (FM) has gained significant attention as a simulation-free generative model. This paper discusses the convergence properties of FM for large sample size under the $p$-Wasserstein distance. We establish that FM can achieve an almost minimax optimal convergence rate for $1 leq p leq 2$, presenting the first theoretical evidence that FM can reach convergence rates comparable to those of diffusion models.
  arXiv  Detail & Related papers  (2024-05-31T14:54:51Z)
• Equivariant Flow Matching with Hybrid Probability Transport [69.11915545210393]
  Diffusion Models (DMs) have demonstrated effectiveness in generating feature-rich geometries. DMs typically suffer from unstable probability dynamics with inefficient sampling speed. We introduce geometric flow matching, which enjoys the advantages of both equivariant modeling and stabilized probability dynamics.
  arXiv  Detail & Related papers  (2023-12-12T11:13:13Z)
• Equivariant flow matching [0.9208007322096533]
  We introduce equivariant flow matching, a new training objective for equivariant continuous normalizing flows (CNFs) Equivariant flow matching exploits the physical symmetries of the target energy for efficient, simulation-free training of equivariant CNFs. Our results show that the equivariant flow matching objective yields flows with shorter integration paths, improved sampling efficiency, and higher scalability compared to existing methods.
  arXiv  Detail & Related papers  (2023-06-26T19:40:10Z)
• Improving and generalizing flow-based generative models with minibatch optimal transport [90.01613198337833]
  We introduce the generalized conditional flow matching (CFM) technique for continuous normalizing flows (CNFs) CFM features a stable regression objective like that used to train the flow in diffusion models but enjoys the efficient inference of deterministic flow models. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference.
  arXiv  Detail & Related papers  (2023-02-01T14:47:17Z)
• Rigid Body Flows for Sampling Molecular Crystal Structures [4.368185344922342]
  We introduce a new type of normalizing flow that is tailored for modeling positions and orientations of multiple objects in three-dimensional space. Our approach is based on two key ideas: first, we define smooth and expressive flows on the group of unit quaternions, which allows us to capture the continuous rotational motion of rigid bodies. We evaluate the method by training Boltzmann generators for two molecular examples, namely the multi-modal density of a tetrahedral system in an external field and the ice XI phase in the TIP4P water model.
  arXiv  Detail & Related papers  (2023-01-26T19:07:40Z)
• An Energy-Based Prior for Generative Saliency [62.79775297611203]
  We propose a novel generative saliency prediction framework that adopts an informative energy-based model as a prior distribution. With the generative saliency model, we can obtain a pixel-wise uncertainty map from an image, indicating model confidence in the saliency prediction. Experimental results show that our generative saliency model with an energy-based prior can achieve not only accurate saliency predictions but also reliable uncertainty maps consistent with human perception.
  arXiv  Detail & Related papers  (2022-04-19T10:51:00Z)
• Smooth Normalizing Flows [0.0]
  We introduce a class of smooth mixture transformations working on both compact intervals and hypertori. We show that such inverses can be computed from forward evaluations via the inverse function theorem. We demonstrate two advantages of such smooth flows: they allow training by force matching to simulation data and can be used as potentials in molecular dynamics simulations.
  arXiv  Detail & Related papers  (2021-10-01T12:27:14Z)
• E(n) Equivariant Normalizing Flows for Molecule Generation in 3D [87.12477361140716]
  This paper introduces a generative model equivariant to Euclidean symmetries: E(n) Equivariant Normalizing Flows (E-NFs) To the best of our knowledge, this is the first likelihood-based deep generative model that generates molecules in 3D.
  arXiv  Detail & Related papers  (2021-05-19T09:28:54Z)
• A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models [93.24030378630175]
  We propose a novel numerical scheme to optimize the gradient flows for learning energy-based models (EBMs) We derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation. Compared with existing schemes, Wasserstein gradient flow is a smoother and near-optimal numerical scheme to approximate real data densities.
  arXiv  Detail & Related papers  (2019-10-31T02:26:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.