Exponential families from a single KL identity

• URL: http://arxiv.org/abs/2604.28036v1
• Date: Thu, 30 Apr 2026 15:48:03 GMT
• Title: Exponential families from a single KL identity
• Abstract summary: Exponential families encompass the distributions central to modern machine learning -- softmax, Gaussians, and Boltzmann distributions.<n>We isolate a simple identity for exponential families that expresses the KL difference $mathrmKL(q | p_|)$ in terms of the log-partition function $A()$ and the moment $_q$.<n>Remarkably, this identity together with the single fact that $mathrmKL geq 0$ suffices, by direct substitution and rearrangement, to derive a cluster of results that are classically obtained
• Score: 7.106986689736826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exponential families encompass the distributions central to modern machine learning -- softmax, Gaussians, and Boltzmann distributions -- and underlie the theory of variational inference, entropy-regularized reinforcement learning, and RLHF. We isolate a simple identity for exponential families that expresses the KL difference $\mathrm{KL}(q \| p_{λ_2}) - \mathrm{KL}(q \| p_{λ_1})$ in terms of the log-partition function $A(λ)$ and the moment $μ_q$. Remarkably, this identity together with the single fact that $\mathrm{KL} \geq 0$ (with equality iff $p = q$) suffices, by direct substitution and rearrangement, to derive a cluster of results that are classically obtained by separate, heavier arguments: a generalized three-point identity for arbitrary reference distributions, Pythagorean theorems for I-projections and reverse I-projections, convexity of the log-partition function, identification of its Legendre dual in KL terms, the Gibbs variational principle, and the explicit optimizer in KL-regularized reward maximization, including the exponential tilting formula underlying entropy-regularized control and RLHF. Beyond these purely algebraic consequences, standard analytic arguments recover the gradient formula for the log-partition function, the Bregman representation of within-family KL divergence, and the surjectivity of the moment map. The note is self-contained.

Related papers

• Information from coincidences [0.42671348654734564]
  We prove a single algebraic mixed coincidence identity derivation that unifies a broad swath of informationtheoretic variational results.<n>Identity yields a unified cornerstones of information theory: concentration of empirical distributions, hypothesis-testing error exponents, change-measure inequalities, and laws governing rare-pattern coincidences.
  arXiv  Detail & Related papers  (2026-06-23T18:01:05Z)
• Optimal Stability of KL Divergence under Gaussian Perturbations [30.190488055856207]
  existing relaxed triangle inequalities for KL divergence critically rely on the assumption that all involved distributions are Gaussian.<n>We establish a sharp stability bound between an arbitrary distribution and Gaussian families under mild moment conditions.<n>We prove that this $sqrt$ rate is optimal in general, even within the Gaussian family.
  arXiv  Detail & Related papers  (2026-04-13T05:49:59Z)
• Regularized Online RLHF with Generalized Bilinear Preferences [68.44113000390544]
  We consider the problem of contextual online RLHF with general preferences.<n>We adopt the Generalized Bilinear Preference Model to capture preferences via low-rank, skew-symmetric matrices.<n>We prove that the dual gap of the greedy policy is bounded by the square of the estimation error.
  arXiv  Detail & Related papers  (2026-02-26T15:27:53Z)
• Value bounds and Convergence Analysis for Averages of LRP attributions [44.992386137813014]
  We analyze numerical properties of Layer-wise relevance propagation (LRP)-type attribution methods by representing them as a product of modified gradient matrices.<n>In particular, our analysis reveals that the constants for LRP-beta remain independent of weight norms, a significant distinction from both gradient-based methods and LRP-epsilon.
  arXiv  Detail & Related papers  (2025-09-10T19:50:00Z)
• Evolution of Gaussians in the Hellinger-Kantorovich-Boltzmann gradient flow [44.500314182711236]
  We derive explicit ordinary equations govern the evolution of mean, covariance, and mass under HK-Boltzmann flow.<n>We demonstrate exponential convergence to equilibrium through constructing a gradient structure associated with a Polyak-Lojasiewicz-type inequalities.<n>We extend our analysis to non-Gaussian targets exhibiting strong log-lambda-concavity.
  arXiv  Detail & Related papers  (2025-04-29T03:54:56Z)
• Logarithmic Regret for Online KL-Regularized Reinforcement Learning [51.113248212150964]
  KL-regularization plays a pivotal role in improving efficiency of RL fine-tuning for large language models.<n>Despite its empirical advantage, the theoretical difference between KL-regularized RL and standard RL remains largely under-explored.<n>We propose an optimistic-based KL-regularized online contextual bandit algorithm, and provide a novel analysis of its regret.
  arXiv  Detail & Related papers  (2025-02-11T11:11:05Z)
• Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians [27.20127082606962]
  Variational inference (VI) is a popular approach in Bayesian inference. This work aims to contribute to the theoretical study of VI in the non-Gaussian case.
  arXiv  Detail & Related papers  (2024-06-06T12:38:59Z)
• Asymptotics of Language Model Alignment [27.37118975691123]
  We show that the optimal KL-constrained RL solution satisfies a large deviation principle. We also show that the rate of growth of the scaled cumulants of the reward is characterized by proper Renyi cross entropy.
  arXiv  Detail & Related papers  (2024-04-02T08:40:07Z)
• Optimal policy evaluation using kernel-based temporal difference methods [78.83926562536791]
  We use kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process. We derive a non-asymptotic upper bound on the error with explicit dependence on the eigenvalues of the associated kernel operator. We prove minimax lower bounds over sub-classes of MRPs.
  arXiv  Detail & Related papers  (2021-09-24T14:48:20Z)
• Optimal Bounds between $f$-Divergences and Integral Probability Metrics [8.401473551081748]
  Families of $f$-divergences and Integral Probability Metrics are widely used to quantify similarity between probability distributions. We systematically study the relationship between these two families from the perspective of convex duality. We obtain new bounds while also recovering in a unified manner well-known results, such as Hoeffding's lemma.
  arXiv  Detail & Related papers  (2020-06-10T17:39:11Z)
• Approximation Schemes for ReLU Regression [80.33702497406632]
  We consider the fundamental problem of ReLU regression. The goal is to output the best fitting ReLU with respect to square loss given to draws from some unknown distribution.
  arXiv  Detail & Related papers  (2020-05-26T16:26:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.