Two tales for a geometric Jensen--Shannon divergence

• URL: http://arxiv.org/abs/2508.05066v1
• Date: Thu, 07 Aug 2025 06:34:39 GMT
• Title: Two tales for a geometric Jensen--Shannon divergence
• Authors: Frank Nielsen,
• Abstract summary: The geometric Jensen--Shannon divergence (G-JSD) gained popularity in machine learning and information sciences.<n>We introduce an alternative definition of the geometric Jensen--Shannon divergence tailored to positive densities.<n>This novel divergence is termed the extended G-JSD as it extends to more general positive measures.
• Score: 7.070726553564701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The geometric Jensen--Shannon divergence (G-JSD) gained popularity in machine learning and information sciences thanks to its closed-form expression between Gaussian distributions. In this work, we introduce an alternative definition of the geometric Jensen--Shannon divergence tailored to positive densities which does not normalize geometric mixtures. This novel divergence is termed the extended G-JSD as it extends to more general positive measures. We give explicitly the gap between the extended G-JSD and G-JSD when considering probability densities, and report both lower and upper bounds in terms of other statistical divergences. We derive corresponding closed-form expressions when considering the case of multivariate Gaussian distributions often met in applications. Finally, we show that these two types of geometric JSDs, the G-JSD and the extended G-JSD, can be interpreted as regularizations of the ordinary JSD by additive terms.

Related papers

• Geometric Gaussian Approximations of Probability Distributions [25.021782278452005]
  We study the expressivity of geometric Gaussian approximations.<n>We provide a constructive proof that such approximations are universal.<n>We discuss whether a common diffeomorphism can be found to obtain uniformly high-quality geometric Gaussian approximations.
  arXiv  Detail & Related papers  (2025-07-01T09:54:43Z)
• Geometric Jensen-Shannon Divergence Between Gaussian Measures On Hilbert Space [5.985204759362746]
  This work studies the Geometric Jensen-Shannon divergence, based on the notion of geometric mean of probability measures, in the setting of Gaussian measures on an infinite-dimensional Hilbert space.<n>We present a closed form expression for this divergence that directly generalizes the finite-dimensional version.
  arXiv  Detail & Related papers  (2025-06-12T08:53:16Z)
• Bridging Geometric States via Geometric Diffusion Bridge [79.60212414973002]
  We introduce the Geometric Diffusion Bridge (GDB), a novel generative modeling framework that accurately bridges initial and target geometric states. GDB employs an equivariant diffusion bridge derived by a modified version of Doob's $h$-transform for connecting geometric states. We show that GDB surpasses existing state-of-the-art approaches, opening up a new pathway for accurately bridging geometric states.
  arXiv  Detail & Related papers  (2024-10-31T17:59:53Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions.<n>For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling.<n>We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach [49.97755400231656]
  We show that a new accelerated DDPM sampler achieves accelerated performance for three broad distribution classes not considered before.<n>Our results show an improved dependency on the data dimension $d$ among accelerated DDPM type samplers.
  arXiv  Detail & Related papers  (2024-02-21T16:11:47Z)
• Scaling Riemannian Diffusion Models [68.52820280448991]
  We show that our method enables us to scale to high dimensional tasks on nontrivial manifold. We model QCD densities on $SU(n)$ lattices and contrastively learned embeddings on high dimensional hyperspheres.
  arXiv  Detail & Related papers  (2023-10-30T21:27:53Z)
• The Representation Jensen-Shannon Divergence [0.0]
  Quantifying the difference between probability distributions is crucial in machine learning. This work proposes the representation Jensen-Shannon divergence (RJSD), a novel measure inspired by the traditional Jensen-Shannon divergence. Our results demonstrate RJSD's superiority in two-sample testing, distribution shift detection, and unsupervised domain adaptation.
  arXiv  Detail & Related papers  (2023-05-25T19:44:36Z)
• Hyperbolic VAE via Latent Gaussian Distributions [7.258394470200572]
  We propose a Gaussian manifold variational auto-encoder (GM-VAE) whose latent space consists of a set of Gaussian distributions. In experiments, we demonstrate the efficacy of GM-VAE on two different tasks: density estimation of image datasets and environment modeling in model-based reinforcement learning.
  arXiv  Detail & Related papers  (2022-09-30T04:09:06Z)
• The Schr\"odinger Bridge between Gaussian Measures has a Closed Form [101.79851806388699]
  We focus on the dynamic formulation of OT, also known as the Schr"odinger bridge (SB) problem. In this paper, we provide closed-form expressions for SBs between Gaussian measures.
  arXiv  Detail & Related papers  (2022-02-11T15:59:01Z)
• Riemannian Score-Based Generative Modeling [56.20669989459281]
  We introduce score-based generative models (SGMs) demonstrating remarkable empirical performance. Current SGMs make the underlying assumption that the data is supported on a Euclidean manifold with flat geometry. This prevents the use of these models for applications in robotics, geoscience or protein modeling.
  arXiv  Detail & Related papers  (2022-02-06T11:57:39Z)
• Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities [16.069404547401373]
  The Jeffreys divergence is a renown symmetrization of the statistical Kullback-Leibler which is often used in machine learning, signal processing, and information sciences. We propose a simple yet fastarine to approximate the Jeffreys divergence between two GMMs of arbitrary number of components.
  arXiv  Detail & Related papers  (2021-07-13T07:58:01Z)
• Entropy-Regularized $2$-Wasserstein Distance between Gaussian Measures [2.320417845168326]
  We study the Gaussian geometry under the entropy-regularized 2-Wasserstein distance. We provide a fixed-point characterization of a population barycenter when restricted to the manifold of Gaussians. As the geometries change by varying the regularization magnitude, we study the limiting cases of vanishing and infinite magnitudes.
  arXiv  Detail & Related papers  (2020-06-05T13:18:57Z)
• Discriminator Contrastive Divergence: Semi-Amortized Generative Modeling by Exploring Energy of the Discriminator [85.68825725223873]
  Generative Adversarial Networks (GANs) have shown great promise in modeling high dimensional data. We introduce the Discriminator Contrastive Divergence, which is well motivated by the property of WGAN's discriminator. We demonstrate the benefits of significant improved generation on both synthetic data and several real-world image generation benchmarks.
  arXiv  Detail & Related papers  (2020-04-05T01:50:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.