Wasserstein KL-divergence for Gaussian distributions

• URL: http://arxiv.org/abs/2503.24022v1
• Date: Mon, 31 Mar 2025 12:49:01 GMT
• Title: Wasserstein KL-divergence for Gaussian distributions
• Authors: Adwait Datar, Nihat Ay,
• Abstract summary: We show that this version is consistent with the geometry of the sample space $Bbb Rn$.<n>In particular, we can evaluate the WKL-divergence of the Dirac measures concentrated in two points which turns out to be proportional to the squared distance between these points.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a new version of the KL-divergence for Gaussian distributions which is based on Wasserstein geometry and referred to as WKL-divergence. We show that this version is consistent with the geometry of the sample space ${\Bbb R}^n$. In particular, we can evaluate the WKL-divergence of the Dirac measures concentrated in two points which turns out to be proportional to the squared distance between these points.

Related papers

• Better Estimation of the KL Divergence Between Language Models [58.7977683502207]
  Estimating the Kullback--Leibler (KL) divergence between language models has many applications. We introduce a Rao--Blackwellized estimator that is also unbiased and provably has variance less than or equal to that of the standard Monte Carlo estimator.
  arXiv  Detail & Related papers  (2025-04-14T18:40:02Z)
• Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergence [7.273481485032721]
  We study the statistical and geometrical properties of the Kullback-Leibler divergence with kernel covariance operators introduced by Bach [2022]<n>Unlike the classical Kullback-Leibler (KL) divergence that involves density ratios, the KKL compares probability distributions through covariance operators (embeddings) in a reproducible kernel Hilbert space (RKHS)<n>This novel divergence hence shares parallel but different aspects with both the standard Kullback-Leibler between probability distributions and kernel embeddings metrics such as the maximum mean discrepancy.
  arXiv  Detail & Related papers  (2024-08-29T14:01:30Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions.<n>For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling.<n>We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space [19.19325201882727]
  Variational inference (VI) seeks to approximate a target distribution $pi$ by an element of a tractable family of distributions. We develop the Forward-Backward Gaussian Variational Inference (FB-GVI) algorithm to solve Gaussian VI. For our proposed algorithm, we obtain state-of-the-art convergence guarantees when $pi$ is log-smooth and log-concave.
  arXiv  Detail & Related papers  (2023-04-10T19:49:50Z)
• Concentration Bounds for Discrete Distribution Estimation in KL Divergence [21.640337031842368]
  We show that the deviation from mean scales as $sqrtk/n$ when $n ge k$ improves upon the best prior result of $k/n$. We also establish a matching lower bound that shows that our bounds are tight up to polylogarithmic factors.
  arXiv  Detail & Related papers  (2023-02-14T07:17:19Z)
• Wrapped Distributions on homogeneous Riemannian manifolds [58.720142291102135]
  Control over distributions' properties, such as parameters, symmetry and modality yield a family of flexible distributions. We empirically validate our approach by utilizing our proposed distributions within a variational autoencoder and a latent space network model.
  arXiv  Detail & Related papers  (2022-04-20T21:25:21Z)
• A Note on Optimizing Distributions using Kernel Mean Embeddings [94.96262888797257]
  Kernel mean embeddings represent probability measures by their infinite-dimensional mean embeddings in a reproducing kernel Hilbert space. We show that when the kernel is characteristic, distributions with a kernel sum-of-squares density are dense. We provide algorithms to optimize such distributions in the finite-sample setting.
  arXiv  Detail & Related papers  (2021-06-18T08:33:45Z)
• KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support [27.165565512841656]
  We study the gradient flow for a relaxed approximation to the Kullback-Leibler divergence between a moving source and a fixed target distribution. This approximation, termed the KALE (KL approximate lower-bound estimator), solves a regularized version of the Fenchel dual problem defining the KL over a restricted class of functions.
  arXiv  Detail & Related papers  (2021-06-16T16:37:43Z)
• Exact Recovery in the General Hypergraph Stochastic Block Model [92.28929858529679]
  This paper investigates fundamental limits of exact recovery in the general d-uniform hypergraph block model (d-HSBM) We show that there exists a sharp threshold such that exact recovery is achievable above the threshold and impossible below it.
  arXiv  Detail & Related papers  (2021-05-11T03:39:08Z)
• $\alpha$-Geodesical Skew Divergence [5.3556221126231085]
  The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter $lambda$, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be absolutely continuous with respect to the source distribution.
  arXiv  Detail & Related papers  (2021-03-31T13:27:58Z)
• A diffusion approach to Stein's method on Riemannian manifolds [65.36007959755302]
  We exploit the relationship between the generator of a diffusion on $mathbf M$ with target invariant measure and its characterising Stein operator. We derive Stein factors, which bound the solution to the Stein equation and its derivatives. We imply that the bounds for $mathbb Rm$ remain valid when $mathbf M$ is a flat manifold.
  arXiv  Detail & Related papers  (2020-03-25T17:03:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.