Curved representational Bregman divergences and their applications

• URL: http://arxiv.org/abs/2504.05654v2
• Date: Wed, 25 Jun 2025 07:53:44 GMT
• Title: Curved representational Bregman divergences and their applications
• Authors: Frank Nielsen,
• Abstract summary: We show that the barycenter of a finite weighted set of parameters under a curved Bregman divergence amounts to the right Bregman projection onto the nonlinear subspace of the barycenter.<n>As an application, we report an efficient method to calculate the intersection of a finite set of $alpha$-divergence spheres.
• Score: 7.070726553564701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: By analogy to curved exponential families in statistics, we define curved Bregman divergences as Bregman divergences restricted to nonlinear parameter subspaces. We show that the barycenter of a finite weighted set of parameters under a curved Bregman divergence amounts to the right Bregman projection onto the nonlinear subspace of the barycenter with respect to the full Bregman divergence. We demonstrate the significance of curved Bregman divergences with two examples: (1) symmetrized Bregman divergences and (2) the Kullback-Leibler divergence between circular complex normal distributions. We then consider monotonic embeddings to define representational curved Bregman divergences and show that the $\alpha$-divergences are representational curved Bregman divergences with respect to $\alpha$-embeddings of the probability simplex into the positive measure cone. As an application, we report an efficient method to calculate the intersection of a finite set of $\alpha$-divergence spheres.

Related papers

• A Connection Between Learning to Reject and Bhattacharyya Divergences [57.942664964198286]
  We consider learning a joint ideal distribution over both inputs and labels.<n>We develop a link between rejection and thresholding different statistical divergences.<n>In general, we find that rejecting via a Bhattacharyya divergence is less aggressive than Chow's Rule.
  arXiv  Detail & Related papers  (2025-05-08T14:18:42Z)
• pyBregMan: A Python library for Bregman Manifolds [56.880400342980394]
  A Bregman manifold is a dually flat space in information geometry which admits as a canonical divergence a Bregman divergence. We describe the design of pyBregMan, a library which implements generic operations on Bregman manifold. The library also provides several core algorithms for various applications in statistics, machine learning, information fusion, and so on.
  arXiv  Detail & Related papers  (2024-08-08T02:38:19Z)
• Sampling and estimation on manifolds using the Langevin diffusion [45.57801520690309]
  Two estimators of linear functionals of $mu_phi $ based on the discretized Markov process are considered. Error bounds are derived for sampling and estimation using a discretization of an intrinsically defined Langevin diffusion.
  arXiv  Detail & Related papers  (2023-12-22T18:01:11Z)
• Divergences induced by dual subtractive and divisive normalizations of exponential families and their convex deformations [7.070726553564701]
  We show that skewed Bhattacharryya distances between probability densities of an exponential family amounts to skewed Jensen divergences induced by the cumulant function. We then show how comparative convexity with respect to a pair of quasi-arithmetic means allows to deform both convex functions and their arguments.
  arXiv  Detail & Related papers  (2023-12-20T08:59:05Z)
• Intrinsic Bayesian Cramér-Rao Bound with an Application to Covariance Matrix Estimation [49.67011673289242]
  This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a smooth manifold. It induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure.
  arXiv  Detail & Related papers  (2023-11-08T15:17:13Z)
• Planar Curve Registration using Bayesian Inversion [0.0]
  We study parameterisation-independent closed curve matching as a Bayesian inverse problem. The motion of the curve is modelled via a curve on the diffeomorphism group acting on the ambient space. We adopt ensemble Kalman inversion using a negative Sobolev mismatch penalty to measure the discrepancy between the target and the ensemble mean shape.
  arXiv  Detail & Related papers  (2023-07-10T21:26:43Z)
• Curvature-Independent Last-Iterate Convergence for Games on Riemannian Manifolds [77.4346324549323]
  We show that a step size agnostic to the curvature of the manifold achieves a curvature-independent and linear last-iterate convergence rate. To the best of our knowledge, the possibility of curvature-independent rates and/or last-iterate convergence has not been considered before.
  arXiv  Detail & Related papers  (2023-06-29T01:20:44Z)
• Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding [12.020235141059992]
  We show that the arithmetic mean over arguments minimizes the expected Bregman divergence to a single representative point. Our analysis highlights the interplay between quasi-arithmetic means, parametric families, and divergence functionals.
  arXiv  Detail & Related papers  (2022-09-15T17:22:04Z)
• Understanding the bias-variance tradeoff of Bregman divergences [13.006468721874372]
  This paper builds upon the work of Pfau (2013), which generalized the bias variance tradeoff to any Bregman divergence loss function. We show that, similarly to the label, the central prediction can be interpreted as the mean of a random variable, where the mean operates in a dual space defined by the loss function itself.
  arXiv  Detail & Related papers  (2022-02-08T22:06:16Z)
• A Stochastic Bregman Primal-Dual Splitting Algorithm for Composite Optimization [2.9112649816695204]
  We study a first order primal-dual method for solving convex-concave saddle point problems over real Banach spaces. Our framework is general and does not need strong convexity of the entropies inducing the Bregman divergences in the algorithm. Numerical applications are considered including entropically regularized Wasserstein barycenter problems and regularized inverse problems on the simplex.
  arXiv  Detail & Related papers  (2021-12-22T14:47:44Z)
• On the Robustness to Misspecification of $\alpha$-Posteriors and Their Variational Approximations [12.52149409594807]
  $alpha$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce the Kullback-Leibler (KL) divergence from the true, but perhaps infeasible, posterior distribution when there is potential parametric model misspecification.
  arXiv  Detail & Related papers  (2021-04-16T19:11:53Z)
• $\alpha$-Geodesical Skew Divergence [5.3556221126231085]
  The asymmetric skew divergence smooths one of the distributions by mixing it, to a degree determined by the parameter $lambda$, with the other distribution. Such divergence is an approximation of the KL divergence that does not require the target distribution to be absolutely continuous with respect to the source distribution.
  arXiv  Detail & Related papers  (2021-03-31T13:27:58Z)
• Continuous Regularized Wasserstein Barycenters [51.620781112674024]
  We introduce a new dual formulation for the regularized Wasserstein barycenter problem. We establish strong duality and use the corresponding primal-dual relationship to parametrize the barycenter implicitly using the dual potentials of regularized transport problems.
  arXiv  Detail & Related papers  (2020-08-28T08:28:06Z)
• On the Theoretical Equivalence of Several Trade-Off Curves Assessing Statistical Proximity [4.626261940793027]
  We propose a unification of four curves known respectively as: the precision-recall (PR) curve, the Lorenz curve, the receiver operating characteristic (ROC) curve and a special case of R'enyi divergence frontiers. In addition, we discuss possible links between PR / Lorenz curves with the derivation of domain adaptation bounds.
  arXiv  Detail & Related papers  (2020-06-21T14:32:38Z)
• Debiased Sinkhorn barycenters [110.79706180350507]
  Entropy regularization in optimal transport (OT) has been the driver of many recent interests for Wasserstein metrics and barycenters in machine learning. We show how this bias is tightly linked to the reference measure that defines the entropy regularizer. We propose debiased Wasserstein barycenters that preserve the best of both worlds: fast Sinkhorn-like iterations without entropy smoothing.
  arXiv  Detail & Related papers  (2020-06-03T23:06:02Z)
• A diffusion approach to Stein's method on Riemannian manifolds [65.36007959755302]
  We exploit the relationship between the generator of a diffusion on $mathbf M$ with target invariant measure and its characterising Stein operator. We derive Stein factors, which bound the solution to the Stein equation and its derivatives. We imply that the bounds for $mathbb Rm$ remain valid when $mathbf M$ is a flat manifold.
  arXiv  Detail & Related papers  (2020-03-25T17:03:58Z)
• Differentiating through the Fr\'echet Mean [51.32291896926807]
  Fr'echet mean is a generalization of the Euclidean mean. We show how to differentiate through the Fr'echet mean for arbitrary Riemannian manifold. This fully integrates the Fr'echet mean into the hyperbolic neural network pipeline.
  arXiv  Detail & Related papers  (2020-02-29T19:49:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.