Annealed Importance Sampling with q-Paths

• URL: http://arxiv.org/abs/2012.07823v1
• Date: Mon, 14 Dec 2020 18:57:05 GMT
• Title: Annealed Importance Sampling with q-Paths
• Authors: Rob Brekelmans, Vaden Masrani, Thang Bui, Frank Wood, Aram Galstyan, Greg Ver Steeg, Frank Nielsen
• Abstract summary: Annealed importance sampling (AIS) is the gold standard for estimating partition functions or marginal likelihoods. Existing literature has been primarily limited to the geometric mixture or moment-averaged paths associated with the exponential family and KL divergence. We explore AIS using $q$-paths, which include the geometric path as a special case and are related to the homogeneous power mean, deformed exponential family, and $alpha$-divergence.
• Score: 51.73925445218365
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Annealed importance sampling (AIS) is the gold standard for estimating partition functions or marginal likelihoods, corresponding to importance sampling over a path of distributions between a tractable base and an unnormalized target. While AIS yields an unbiased estimator for any path, existing literature has been primarily limited to the geometric mixture or moment-averaged paths associated with the exponential family and KL divergence. We explore AIS using $q$-paths, which include the geometric path as a special case and are related to the homogeneous power mean, deformed exponential family, and $\alpha$-divergence.

Related papers

• Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyond [24.86929310909572]
  We show that using the geometric path brings down the estimation error from an exponential to a function of the distance between the target and proposal. We propose a two-step estimator to approximate the optimal path in an efficient way.
  arXiv  Detail & Related papers  (2023-10-05T21:16:55Z)
• Adaptive Annealed Importance Sampling with Constant Rate Progress [68.8204255655161]
  Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution. We propose the Constant Rate AIS algorithm and its efficient implementation for $alpha$-divergences.
  arXiv  Detail & Related papers  (2023-06-27T08:15:28Z)
• Generalized equivalences between subsampling and ridge regularization [3.1346887720803505]
  We prove structural and risk equivalences between subsampling and ridge regularization for ensemble ridge estimators. An indirect implication of our equivalences is that optimally tuned ridge regression exhibits a monotonic prediction risk in the data aspect ratio.
  arXiv  Detail & Related papers  (2023-05-29T14:05:51Z)
• Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding [12.020235141059992]
  We show that the arithmetic mean over arguments minimizes the expected Bregman divergence to a single representative point. Our analysis highlights the interplay between quasi-arithmetic means, parametric families, and divergence functionals.
  arXiv  Detail & Related papers  (2022-09-15T17:22:04Z)
• Matching Normalizing Flows and Probability Paths on Manifolds [57.95251557443005]
  Continuous Normalizing Flows (CNFs) are generative models that transform a prior distribution to a model distribution by solving an ordinary differential equation (ODE) We propose to train CNFs by minimizing probability path divergence (PPD), a novel family of divergences between the probability density path generated by the CNF and a target probability density path. We show that CNFs learned by minimizing PPD achieve state-of-the-art results in likelihoods and sample quality on existing low-dimensional manifold benchmarks.
  arXiv  Detail & Related papers  (2022-07-11T08:50:19Z)
• Robust Estimation for Nonparametric Families via Generative Adversarial Networks [92.64483100338724]
  We provide a framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems. Our work extend these to robust mean estimation, second moment estimation, and robust linear regression. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance.
  arXiv  Detail & Related papers  (2022-02-02T20:11:33Z)
• Nonconvex Stochastic Scaled-Gradient Descent and Generalized Eigenvector Problems [98.34292831923335]
  Motivated by the problem of online correlation analysis, we propose the emphStochastic Scaled-Gradient Descent (SSD) algorithm. We bring these ideas together in an application to online correlation analysis, deriving for the first time an optimal one-time-scale algorithm with an explicit rate of local convergence to normality.
  arXiv  Detail & Related papers  (2021-12-29T18:46:52Z)
• q-Paths: Generalizing the Geometric Annealing Path using Power Means [51.73925445218366]
  We introduce $q$-paths, a family of paths which includes the geometric and arithmetic mixtures as special cases. We show that small deviations away from the geometric path yield empirical gains for Bayesian inference.
  arXiv  Detail & Related papers  (2021-07-01T21:09:06Z)
• Cram\'er-Rao Lower Bounds Arising from Generalized Csisz\'ar Divergences [17.746238062801293]
  We study the geometry of probability distributions with respect to a generalized family of Csisz'ar $f$-divergences. We show that these formulations lead us to find unbiased and efficient estimators for the escort model.
  arXiv  Detail & Related papers  (2020-01-14T13:41:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.