Particle Dynamics for Learning EBMs
- URL: http://arxiv.org/abs/2111.13772v1
- Date: Fri, 26 Nov 2021 23:41:07 GMT
- Title: Particle Dynamics for Learning EBMs
- Authors: Kirill Neklyudov, Priyank Jaini, Max Welling
- Abstract summary: Energy-based modeling is a promising approach to unsupervised learning, which yields many downstream applications from a single model.
The main difficulty in learning energy-based models with the "contrastive approaches" is the generation of samples from the current energy function at each iteration.
This paper proposes an alternative approach to getting these samples and avoiding crude MCMC sampling from the current model.
- Score: 83.59335980576637
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Energy-based modeling is a promising approach to unsupervised learning, which
yields many downstream applications from a single model. The main difficulty in
learning energy-based models with the "contrastive approaches" is the
generation of samples from the current energy function at each iteration. Many
advances have been made to accomplish this subroutine cheaply. Nevertheless,
all such sampling paradigms run MCMC targeting the current model, which
requires infinitely long chains to generate samples from the true energy
distribution and is problematic in practice. This paper proposes an alternative
approach to getting these samples and avoiding crude MCMC sampling from the
current model. We accomplish this by viewing the evolution of the modeling
distribution as (i) the evolution of the energy function, and (ii) the
evolution of the samples from this distribution along some vector field. We
subsequently derive this time-dependent vector field such that the particles
following this field are approximately distributed as the current density
model. Thereby we match the evolution of the particles with the evolution of
the energy function prescribed by the learning procedure. Importantly, unlike
Monte Carlo sampling, our method targets to match the current distribution in a
finite time. Finally, we demonstrate its effectiveness empirically compared to
MCMC-based learning methods.
Related papers
Err
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.