Improved Stein Variational Gradient Descent with Importance Weights

• URL: http://arxiv.org/abs/2210.00462v2
• Date: Tue, 4 Oct 2022 10:41:12 GMT
• Title: Improved Stein Variational Gradient Descent with Importance Weights
• Authors: Lukang Sun and Peter Richt\'arik
• Abstract summary: Stein Variational Gradient Descent (SVGD) is a popular sampling algorithm used in various machine learning tasks. We propose to enhance SVGD via the introduction of importance weights, which leads to a new method for which we coin the name $beta$-SVGD.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stein Variational Gradient Descent (SVGD) is a popular sampling algorithm used in various machine learning tasks. It is well known that SVGD arises from a discretization of the kernelized gradient flow of the Kullback-Leibler divergence $D_{KL}\left(\cdot\mid\pi\right)$, where $\pi$ is the target distribution. In this work, we propose to enhance SVGD via the introduction of importance weights, which leads to a new method for which we coin the name $\beta$-SVGD. In the continuous time and infinite particles regime, the time for this flow to converge to the equilibrium distribution $\pi$, quantified by the Stein Fisher information, depends on $\rho_0$ and $\pi$ very weakly. This is very different from the kernelized gradient flow of Kullback-Leibler divergence, whose time complexity depends on $D_{KL}\left(\rho_0\mid\pi\right)$. Under certain assumptions, we provide a descent lemma for the population limit $\beta$-SVGD, which covers the descent lemma for the population limit SVGD when $\beta\to 0$. We also illustrate the advantages of $\beta$-SVGD over SVGD by simple experiments.

Related papers

• Stein Variational Evolution Strategies [17.315583101484147]
  Stein Variational Gradient Descent (SVGD) is a highly efficient method to sample from an unnormalized probability distribution. Existing gradient-free versions of SVGD make use of simple Monte Carlo approximations or gradients from surrogate distributions, both with limitations. We combine SVGD steps with evolution strategy (ES) updates to improve gradient-free Stein variational inference.
  arXiv  Detail & Related papers  (2024-10-14T11:24:41Z)
• Improved Algorithm for Adversarial Linear Mixture MDPs with Bandit Feedback and Unknown Transition [71.33787410075577]
  We study reinforcement learning with linear function approximation, unknown transition, and adversarial losses. We propose a new algorithm that attains an $widetildeO(dsqrtHS3K + sqrtHSAK)$ regret with high probability.
  arXiv  Detail & Related papers  (2024-03-07T15:03:50Z)
• Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation [9.065034043031668]
  Stein Variational Gradient Descent (SVGD) is a popular variational inference which simulates an interacting particle system to approximately sample from a target distribution. We introduce the notion of virtual particles and develop novel approximations of population-limit dynamics in the space of probability measures. We show that the $n$ particles output by VP-SVGD and GB-SVGD, run for $T$ steps with batch-size $K$, are as good as i.i.i.d samples from a distribution whose Kernel Stein Discrepancy to the target is at most $Oleft(tfrac
  arXiv  Detail & Related papers  (2023-05-27T19:21:28Z)
• Depth Dependence of $\mu$P Learning Rates in ReLU MLPs [72.14317069090407]
  We study the dependence on $n$ and $L$ of the maximal update ($mu$P) learning rate. We find that it has a non-trivial dependence of $L$, scaling like $L-3/2.$
  arXiv  Detail & Related papers  (2023-05-13T01:10:49Z)
• Horizon-Free and Variance-Dependent Reinforcement Learning for Latent Markov Decision Processes [62.90204655228324]
  We study regret minimization for reinforcement learning (RL) in Latent Markov Decision Processes (LMDPs) with context in hindsight. We design a novel model-based algorithmic framework which can be instantiated with both a model-optimistic and a value-optimistic solver.
  arXiv  Detail & Related papers  (2022-10-20T21:32:01Z)
• A Note on the Convergence of Mirrored Stein Variational Gradient Descent under $(L_0,L_1)-$Smoothness Condition [0.0]
  We establish a descent lemma for the population limit Mirrored Stein Variational Gradient Method(MSVGD) This descent lemma does not rely on the path information of MSVGD but rather on a simple assumption for the mirrored distribution $nablaPsi_#piproptoexp(-V)$.
  arXiv  Detail & Related papers  (2022-06-20T11:04:18Z)
• Convergence of Stein Variational Gradient Descent under a Weaker Smoothness Condition [0.0]
  Stein Variational Gradient Descent (SVGD) is an important alternative to the Langevin-type algorithms for sampling from probability distributions. In the existing theory of Langevin-type algorithms and SVGD, the potential function $V$ is often assumed to be $L$-smooth.
  arXiv  Detail & Related papers  (2022-06-01T14:08:35Z)
• Variance-Aware Confidence Set: Variance-Dependent Bound for Linear Bandits and Horizon-Free Bound for Linear Mixture MDP [76.94328400919836]
  We show how to construct variance-aware confidence sets for linear bandits and linear mixture Decision Process (MDP) For linear bandits, we obtain an $widetildeO(mathrmpoly(d)sqrt1 + sum_i=1Ksigma_i2) regret bound, where $d is the feature dimension. For linear mixture MDP, we obtain an $widetildeO(mathrmpoly(d)sqrtK)$ regret bound, where
  arXiv  Detail & Related papers  (2021-01-29T18:57:52Z)
• Kernel Stein Generative Modeling [68.03537693810972]
  Gradient Langevin Dynamics (SGLD) demonstrates impressive results with energy-based models on high-dimensional and complex data distributions. Stein Variational Gradient Descent (SVGD) is a deterministic sampling algorithm that iteratively transports a set of particles to approximate a given distribution. We propose noise conditional kernel SVGD (NCK-SVGD), that works in tandem with the recently introduced Noise Conditional Score Network estimator.
  arXiv  Detail & Related papers  (2020-07-06T21:26:04Z)
• A Non-Asymptotic Analysis for Stein Variational Gradient Descent [44.30569261307296]
  We provide a novel finite time analysis for the Stein Variational Gradient Descent algorithm. We provide a descent lemma establishing that the algorithm decreases the objective at each iteration. We also provide a convergence result of the finite particle system corresponding to the practical implementation of SVGD to its population version.
  arXiv  Detail & Related papers  (2020-06-17T12:01:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.