Learning with Importance Weighted Variational Inference: Asymptotics for Gradient Estimators of the VR-IWAE Bound

• URL: http://arxiv.org/abs/2410.12035v1
• Date: Tue, 15 Oct 2024 20:09:06 GMT
• Title: Learning with Importance Weighted Variational Inference: Asymptotics for Gradient Estimators of the VR-IWAE Bound
• Authors: Kamélia Daudel, François Roueff,
• Abstract summary: Several popular variational bounds involving importance weighting ideas have been proposed to generalize and improve on the Evidence Lower BOund. The VR-IWAE bound was introduced as a variational bound that unifies the ELBO, IWAE and VR bounds methodologies.
• Score: 3.115375810642661
• License:
• Abstract: Several popular variational bounds involving importance weighting ideas have been proposed to generalize and improve on the Evidence Lower BOund (ELBO) in the context of maximum likelihood optimization, such as the Importance Weighted Auto-Encoder (IWAE) and the Variational R\'enyi (VR) bounds. The methodology to learn the parameters of interest using these bounds typically amounts to running gradient-based variational inference algorithms that incorporate the reparameterization trick. However, the way the choice of the variational bound impacts the outcome of variational inference algorithms can be unclear. Recently, the VR-IWAE bound was introduced as a variational bound that unifies the ELBO, IWAE and VR bounds methodologies. In this paper, we provide two analyses for the reparameterized and doubly-reparameterized gradient estimators of the VR-IWAE bound, which reveal the advantages and limitations of these gradient estimators while enabling us to compare of the ELBO, IWAE and VR bounds methodologies. Our work advances the understanding of importance weighted variational inference methods and we illustrate our theoretical findings empirically.

Related papers

• Geometric Remove-and-Retrain (GOAR): Coordinate-Invariant eXplainable AI Assessment [3.7399138244928145]
  Remove-and-Retrain (ROAR) is a widely accepted approach for assessing the importance of individual pixels. We introduce an alternative feature-perturbation approach named Geometric Remove-and-Retrain (GOAR) GOAR transcends the limitations of pixel-centric metrics.
  arXiv  Detail & Related papers  (2024-07-17T08:28:53Z)
• Generalized Exponentiated Gradient Algorithms and Their Application to On-Line Portfolio Selection [13.075438411817117]
  This paper introduces a novel family of generalized exponentiated gradient (EG) updates derived from an Alpha-Beta divergence regularization function. As an illustration of their applicability, we evaluate the proposed updates in addressing the online portfolio selection problem (OLPS) using gradient-based methods.
  arXiv  Detail & Related papers  (2024-06-02T07:56:30Z)
• Latent Variable Representation for Reinforcement Learning [131.03944557979725]
  It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning. We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle. In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
  arXiv  Detail & Related papers  (2022-12-17T00:26:31Z)
• Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics [21.51266421854127]
  We formalize and study the VR-IWAE bound, a generalization of the Importance Weighted Auto-Encoder (IWAE) bound. We show that the VR-IWAE bound enjoys several desirable properties and notably leads to the same gradient descent procedure as the VR bound. We then provide two complementary theoretical analyses of the VR-IWAE bound and thus of the standard IWAE bound.
  arXiv  Detail & Related papers  (2022-10-12T14:15:39Z)
• Variational Inference with Holder Bounds [68.8008396694788]
  We present a careful analysis of the thermodynamic variational objective (TVO) We reveal how the pathological geometry of thermodynamic curves negatively affects TVO. This motivates our new VI objectives, named the Holder bounds, which flatten the thermodynamic curves and promise to achieve a one-step approximation of the exact marginal log-likelihood.
  arXiv  Detail & Related papers  (2021-11-04T15:35:47Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Boosting Variational Inference With Locally Adaptive Step-Sizes [27.122745595473383]
  Boosting Variational Inference allows practitioners to obtain increasingly good posterior approximations by spending more compute. The main obstacle to widespread adoption of Boosting Variational Inference is the amount of resources necessary to improve over a strong Variational Inference baseline. We describe how the global curvature impacts time and memory consumption, address the problem with the notion of local curvature, and provide a novel approximate backtracking algorithm for estimating local curvature.
  arXiv  Detail & Related papers  (2021-05-19T16:41:33Z)
• Quantized Variational Inference [6.09170287691728]
  We show how Quantized Variational Inference produces variance free gradients for ELBO optimization. We show that using Quantized Variational Inference framework leads to fast convergence for both score function and reparametrized gradient.
  arXiv  Detail & Related papers  (2020-11-04T13:22:50Z)
• Adaptive Gradient Method with Resilience and Momentum [120.83046824742455]
  We propose an Adaptive Gradient Method with Resilience and Momentum (AdaRem) AdaRem adjusts the parameter-wise learning rate according to whether the direction of one parameter changes in the past is aligned with the direction of the current gradient. Our method outperforms previous adaptive learning rate-based algorithms in terms of the training speed and the test error.
  arXiv  Detail & Related papers  (2020-10-21T14:49:00Z)
• Disentangling Adaptive Gradient Methods from Learning Rates [65.0397050979662]
  We take a deeper look at how adaptive gradient methods interact with the learning rate schedule. We introduce a "grafting" experiment which decouples an update's magnitude from its direction. We present some empirical and theoretical retrospectives on the generalization of adaptive gradient methods.
  arXiv  Detail & Related papers  (2020-02-26T21:42:49Z)
• Discrete Action On-Policy Learning with Action-Value Critic [72.20609919995086]
  Reinforcement learning (RL) in discrete action space is ubiquitous in real-world applications, but its complexity grows exponentially with the action-space dimension. We construct a critic to estimate action-value functions, apply it on correlated actions, and combine these critic estimated action values to control the variance of gradient estimation. These efforts result in a new discrete action on-policy RL algorithm that empirically outperforms related on-policy algorithms relying on variance control techniques.
  arXiv  Detail & Related papers  (2020-02-10T04:23:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.