Learning Energy-Based Generative Models via Potential Flow: A Variational Principle Approach to Probability Density Homotopy Matching

• URL: http://arxiv.org/abs/2504.16262v1
• Date: Tue, 22 Apr 2025 20:39:07 GMT
• Title: Learning Energy-Based Generative Models via Potential Flow: A Variational Principle Approach to Probability Density Homotopy Matching
• Authors: Junn Yong Loo, Michelle Adeline, Julia Kaiwen Lau, Fang Yu Leong, Hwa Hui Tew, Arghya Pal, Vishnu Monn Baskaran, Chee-Ming Ting, Raphaël C. -W. Phan,
• Abstract summary: Energy-based models (EBMs) are a powerful class of probabilistic generative models.<n>We propose Variational Potential Flow Bayes (VPFB), a new energy-based generative framework.
• Score: 9.12119858170289
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Energy-based models (EBMs) are a powerful class of probabilistic generative models due to their flexibility and interpretability. However, relationships between potential flows and explicit EBMs remain underexplored, while contrastive divergence training via implicit Markov chain Monte Carlo (MCMC) sampling is often unstable and expensive in high-dimensional settings. In this paper, we propose Variational Potential Flow Bayes (VPFB), a new energy-based generative framework that eliminates the need for implicit MCMC sampling and does not rely on auxiliary networks or cooperative training. VPFB learns an energy-parameterized potential flow by constructing a flow-driven density homotopy that is matched to the data distribution through a variational loss minimizing the Kullback-Leibler divergence between the flow-driven and marginal homotopies. This principled formulation enables robust and efficient generative modeling while preserving the interpretability of EBMs. Experimental results on image generation, interpolation, out-of-distribution detection, and compositional generation confirm the effectiveness of VPFB, showing that our method performs competitively with existing approaches in terms of sample quality and versatility across diverse generative modeling tasks.

Related papers

• Energy Matching: Unifying Flow Matching and Energy-Based Models for Generative Modeling [4.584647857042494]
  Generative models often map noise to data by matching flows or scores, but these approaches become cumbersome for incorporating partial observations or additional priors.<n>Inspired by recent advances in Wasserstein gradient flows, we propose Energy Matching, a framework that unifies flow-based approaches with the flexibility of energy-based models (EBMs)<n>We parameterize this dynamic with a single time-independent scalar field, which serves as both a powerful generator and a flexible prior for effective regularization of inverse problems.
  arXiv  Detail & Related papers  (2025-04-14T18:10:58Z)
• Hessian-Informed Flow Matching [4.542719108171107]
  Hessian-Informed Flow Matching is a novel approach that integrates the Hessian of an energy function into conditional flows. This integration allows HI-FM to account for local curvature and anisotropic covariance structures. Empirical evaluations on the MNIST and Lennard-Jones particles datasets demonstrate that HI-FM improves the likelihood of test samples.
  arXiv  Detail & Related papers  (2024-10-15T09:34:52Z)
• Variational Potential Flow: A Novel Probabilistic Framework for Energy-Based Generative Modelling [10.926841288976684]
  We present a novel energy-based generative framework, Variational Potential Flow (VAPO) VAPO aims to learn a potential energy function whose gradient (flow) guides the prior samples, so that their density evolution closely follows an approximate data likelihood homotopy. Images can be generated after training the potential energy, by initializing the samples from Gaussian prior and solving the ODE governing the potential flow on a fixed time interval.
  arXiv  Detail & Related papers  (2024-07-21T18:08:12Z)
• Extended Flow Matching: a Method of Conditional Generation with Generalized Continuity Equation [19.71452214879951]
  conditional generation is one of the most important applications of generative models. We show that we can introduce inductive bias to the conditional generation through the matrix field. We will present our theory along with experimental results that support the competitiveness of EFM in conditional generation.
  arXiv  Detail & Related papers  (2024-02-29T04:12:32Z)
• Equivariant Flow Matching with Hybrid Probability Transport [69.11915545210393]
  Diffusion Models (DMs) have demonstrated effectiveness in generating feature-rich geometries. DMs typically suffer from unstable probability dynamics with inefficient sampling speed. We introduce geometric flow matching, which enjoys the advantages of both equivariant modeling and stabilized probability dynamics.
  arXiv  Detail & Related papers  (2023-12-12T11:13:13Z)
• Learning Energy-Based Prior Model with Diffusion-Amortized MCMC [89.95629196907082]
  Common practice of learning latent space EBMs with non-convergent short-run MCMC for prior and posterior sampling is hindering the model from further progress. We introduce a simple but effective diffusion-based amortization method for long-run MCMC sampling and develop a novel learning algorithm for the latent space EBM based on it.
  arXiv  Detail & Related papers  (2023-10-05T00:23:34Z)
• Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood [64.95663299945171]
  Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming. There exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. We propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs.
  arXiv  Detail & Related papers  (2023-09-10T22:05:24Z)
• On Feature Diversity in Energy-based Models [98.78384185493624]
  An energy-based model (EBM) is typically formed of inner-model(s) that learn a combination of the different features to generate an energy mapping for each input configuration. We extend the probably approximately correct (PAC) theory of EBMs and analyze the effect of redundancy reduction on the performance of EBMs.
  arXiv  Detail & Related papers  (2023-06-02T12:30:42Z)
• Latent Diffusion Energy-Based Model for Interpretable Text Modeling [104.85356157724372]
  We introduce a novel symbiosis between the diffusion models and latent space EBMs in a variational learning framework. We develop a geometric clustering-based regularization jointly with the information bottleneck to further improve the quality of the learned latent space.
  arXiv  Detail & Related papers  (2022-06-13T03:41:31Z)
• Training Deep Energy-Based Models with f-Divergence Minimization [113.97274898282343]
  Deep energy-based models (EBMs) are very flexible in distribution parametrization but computationally challenging. We propose a general variational framework termed f-EBM to train EBMs using any desired f-divergence. Experimental results demonstrate the superiority of f-EBM over contrastive divergence, as well as the benefits of training EBMs using f-divergences other than KL.
  arXiv  Detail & Related papers  (2020-03-06T23:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.