Symplectic Bregman divergences

• URL: http://arxiv.org/abs/2408.12961v3
• Date: Wed, 28 Aug 2024 08:15:18 GMT
• Title: Symplectic Bregman divergences
• Authors: Frank Nielsen,
• Abstract summary: Symplectic Bregman divergences are derived from a symplectic generalization of the Fenchel-Young inequality. Some potential applications of symplectic divergences in geometric mechanics, information geometry, and learning dynamics in machine learning are touched upon.
• Score: 7.070726553564701
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a generalization of Bregman divergences in symplectic vector spaces that we term symplectic Bregman divergences. Symplectic Bregman divergences are derived from a symplectic generalization of the Fenchel-Young inequality which relies on the notion of symplectic subdifferentials. The symplectic Fenchel-Young inequality is obtained using the symplectic Fenchel transform which is defined with respect to the symplectic form. Since symplectic forms can be generically built from pairings of dual systems, we get a generalization of Bregman divergences in dual systems obtained by equivalent symplectic Bregman divergences. In particular, when the symplectic form is derived from an inner product, we show that the corresponding symplectic Bregman divergences amount to ordinary Bregman divergences with respect to composite inner products. Some potential applications of symplectic divergences in geometric mechanics, information geometry, and learning dynamics in machine learning are touched upon.

Related papers

• Curved representational Bregman divergences and their applications [7.070726553564701]
  We define curved Bregman divergences as restrictions of Bregman divergences to sub-dimensional parameter subspaces. We show that the barycenter of a finite weighted parameter set with respect to a curved Bregman divergence amounts to the Bregman projection onto the subspace induced by the constraint of the barycenter.
  arXiv  Detail & Related papers  (2025-04-08T04:05:12Z)
• pyBregMan: A Python library for Bregman Manifolds [56.880400342980394]
  A Bregman manifold is a dually flat space in information geometry which admits as a canonical divergence a Bregman divergence. We describe the design of pyBregMan, a library which implements generic operations on Bregman manifold. The library also provides several core algorithms for various applications in statistics, machine learning, information fusion, and so on.
  arXiv  Detail & Related papers  (2024-08-08T02:38:19Z)
• Understanding Matrix Function Normalizations in Covariance Pooling through the Lens of Riemannian Geometry [63.694184882697435]
  Global Covariance Pooling (GCP) has been demonstrated to improve the performance of Deep Neural Networks (DNNs) by exploiting second-order statistics of high-level representations.
  arXiv  Detail & Related papers  (2024-07-15T07:11:44Z)
• Divergences induced by dual subtractive and divisive normalizations of exponential families and their convex deformations [7.070726553564701]
  We show that skewed Bhattacharryya distances between probability densities of an exponential family amounts to skewed Jensen divergences induced by the cumulant function. We then show how comparative convexity with respect to a pair of quasi-arithmetic means allows to deform both convex functions and their arguments.
  arXiv  Detail & Related papers  (2023-12-20T08:59:05Z)
• Causal Modeling with Stationary Diffusions [89.94899196106223]
  We learn differential equations whose stationary densities model a system's behavior under interventions. We show that they generalize to unseen interventions on their variables, often better than classical approaches. Our inference method is based on a new theoretical result that expresses a stationarity condition on the diffusion's generator in a reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2023-10-26T14:01:17Z)
• A Study of Neural Collapse Phenomenon: Grassmannian Frame, Symmetry and Generalization [91.95109845914267]
  We extend original Neural Collapse Phenomenon by proving Generalized Neural Collapse hypothesis. We obtain Grassmannian Frame structure from the optimization and generalization of classification. We provide a theorem to explain Symmetric Generalization of permutation.
  arXiv  Detail & Related papers  (2023-04-18T11:35:14Z)
• Correspondence between open bosonic systems and stochastic differential equations [77.34726150561087]
  We show that there can also be an exact correspondence at finite $n$ when the bosonic system is generalized to include interactions with the environment. A particular system with the form of a discrete nonlinear Schr"odinger equation is analyzed in more detail.
  arXiv  Detail & Related papers  (2023-02-03T19:17:37Z)
• Mathematical Foundations for a Compositional Account of the Bayesian Brain [0.0]
  We use the tools of contemporary applied category theory to supply functorial semantics for approximate inference. We define fibrations of statistical games and classify various problems of statistical inference as corresponding sections. We construct functors which explain the compositional structure of predictive coding neural circuits under the free energy principle.
  arXiv  Detail & Related papers  (2022-12-23T18:58:17Z)
• Stochastic Mirror Descent in Average Ensemble Models [38.38572705720122]
  The mirror descent (SMD) is a general class of training algorithms, which includes the celebrated gradient descent (SGD) as a special case. In this paper we explore the performance of the mirror potential algorithm on mean-field ensemble models.
  arXiv  Detail & Related papers  (2022-10-27T11:04:00Z)
• Tropical contraction of tensor networks as a Bell inequality optimization toolset [1.1510009152620668]
  We show that finding the classical bound of broad families of Bell inequalities can be naturally framed as the contraction of an associated tensor network. We illustrate our method with paradigmatic examples both in the multipartite scenario and the bipartite scenario with multiple outcomes.
  arXiv  Detail & Related papers  (2022-08-04T17:53:40Z)
• On the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields [93.35534658875731]
  We derive expressions for the Kullback-Leibler divergence between two pairwise isotropic Gaussian-Markov random fields. The proposed equation allows the development of novel similarity measures in image processing and machine learning applications.
  arXiv  Detail & Related papers  (2022-03-24T16:37:24Z)
• Reconciling the Discrete-Continuous Divide: Towards a Mathematical Theory of Sparse Communication [0.0]
  We build rigorous theoretical foundations for discrete/continuous hybrids. We introduce "mixed languages" as strings of hybrid symbols and a new mixed weighted finite state automaton.
  arXiv  Detail & Related papers  (2021-04-01T20:31:13Z)
• The Connection between Discrete- and Continuous-Time Descriptions of Gaussian Continuous Processes [60.35125735474386]
  We show that discretizations yielding consistent estimators have the property of invariance under coarse-graining' This result explains why combining differencing schemes for derivatives reconstruction and local-in-time inference approaches does not work for time series analysis of second or higher order differential equations.
  arXiv  Detail & Related papers  (2021-01-16T17:11:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.