MCMC-Interactive Variational Inference
- URL: http://arxiv.org/abs/2010.02029v1
- Date: Fri, 2 Oct 2020 17:43:20 GMT
- Title: MCMC-Interactive Variational Inference
- Authors: Quan Zhang, Huangjie Zheng, Mingyuan Zhou
- Abstract summary: We propose MCMC-interactive variational inference (MIVI) to estimate the posterior in a time constrained manner.
MIVI takes advantage of the complementary properties of variational inference and MCMC to encourage mutual improvement.
Experiments show that MIVI not only accurately approximates the posteriors but also facilitates designs of gradient MCMC and Gibbs sampling transitions.
- Score: 56.58416764959414
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Leveraging well-established MCMC strategies, we propose MCMC-interactive
variational inference (MIVI) to not only estimate the posterior in a time
constrained manner, but also facilitate the design of MCMC transitions.
Constructing a variational distribution followed by a short Markov chain that
has parameters to learn, MIVI takes advantage of the complementary properties
of variational inference and MCMC to encourage mutual improvement. On one hand,
with the variational distribution locating high posterior density regions, the
Markov chain is optimized within the variational inference framework to
efficiently target the posterior despite a small number of transitions. On the
other hand, the optimized Markov chain with considerable flexibility guides the
variational distribution towards the posterior and alleviates its
underestimation of uncertainty. Furthermore, we prove the optimized Markov
chain in MIVI admits extrapolation, which means its marginal distribution gets
closer to the true posterior as the chain grows. Therefore, the Markov chain
can be used separately as an efficient MCMC scheme. Experiments show that MIVI
not only accurately and efficiently approximates the posteriors but also
facilitates designs of stochastic gradient MCMC and Gibbs sampling transitions.
Related papers
Err
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.