Local Max-Entropy and Free Energy Principles Solved by Belief Propagation

• URL: http://arxiv.org/abs/2207.00841v1
• Date: Sat, 2 Jul 2022 14:20:40 GMT
• Title: Local Max-Entropy and Free Energy Principles Solved by Belief Propagation
• Authors: Olivier Peltre
• Abstract summary: A statistical system is classically defined on a set of microstates $E$ by a global energy function $H : E to mathbbR$, yielding Gibbs probability measures $rhobeta(H)$ for every inverse temperature $beta = T-1$. We show that the generalized belief propagation algorithm solves a collection of local variational principles, by converging to critical points of Bethe-Kikuchi approximations of the free energy $F(beta)$, the Shannon entropy $S(cal U)$, and the variational free energy
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A statistical system is classically defined on a set of microstates $E$ by a global energy function $H : E \to \mathbb{R}$, yielding Gibbs probability measures (softmins) $\rho^\beta(H)$ for every inverse temperature $\beta = T^{-1}$. Gibbs states are simultaneously characterized by free energy principles and the max-entropy principle, with dual constraints on inverse temperature $\beta$ and mean energy ${\cal U}(\beta) = \mathbb{E}_{\rho^\beta}[H]$ respectively. The Legendre transform relates these diverse variational principles which are unfortunately not tractable in high dimension. The global energy is generally given as a sum $H(x) = \sum_{\rm a \subset \Omega} h_{\rm a}(x_{|\rm a})$ of local short-range interactions $h_{\rm a} : E_{\rm a} \to \mathbb{R}$ indexed by bounded subregions ${\rm a} \subset \Omega$, and this local structure can be used to design good approximation schemes on thermodynamic functionals. We show that the generalized belief propagation (GBP) algorithm solves a collection of local variational principles, by converging to critical points of Bethe-Kikuchi approximations of the free energy $F(\beta)$, the Shannon entropy $S(\cal U)$, and the variational free energy ${\cal F}(\beta) = {\cal U} - \beta^{-1} S(\cal U)$, extending an initial correspondence by Yedidia et al. This local form of Legendre duality yields a possible degenerate relationship between mean energy ${\cal U}$ and $\beta$.

Related papers

• Thermodynamics and criticality of supersymmetric spin chains of Haldane-Shastry type [0.0]
  We analyze the thermodynamics and criticality properties of four families of su$(m|n)$ supersymmetric spin chains of Haldane-Shastry (HS) type.
  arXiv  Detail & Related papers  (2024-08-26T17:36:32Z)
• On the $O(\frac{\sqrt{d}}{T^{1/4}})$ Convergence Rate of RMSProp and Its Momentum Extension Measured by $\ell_1$ Norm [59.65871549878937]
  This paper considers the RMSProp and its momentum extension and establishes the convergence rate of $frac1Tsum_k=1T. Our convergence rate matches the lower bound with respect to all the coefficients except the dimension $d$. Our convergence rate can be considered to be analogous to the $frac1Tsum_k=1T.
  arXiv  Detail & Related papers  (2024-02-01T07:21:32Z)
• Thermalization of closed chaotic many-body quantum systems [0.0]
  We investigate thermalization of a chaotic many-body quantum system by combining the Hartree-Fock approach and the Bohigas-Giannoni-Schmit conjecture. We show that in the semiclassical regime, $rm Tr (A rho(t))$ decays with time scale $hbar / Delta$ towards an equilibrium value.
  arXiv  Detail & Related papers  (2023-10-04T11:16:35Z)
• A Unified Framework for Uniform Signal Recovery in Nonlinear Generative Compressed Sensing [68.80803866919123]
  Under nonlinear measurements, most prior results are non-uniform, i.e., they hold with high probability for a fixed $mathbfx*$ rather than for all $mathbfx*$ simultaneously. Our framework accommodates GCS with 1-bit/uniformly quantized observations and single index models as canonical examples. We also develop a concentration inequality that produces tighter bounds for product processes whose index sets have low metric entropy.
  arXiv  Detail & Related papers  (2023-09-25T17:54:19Z)
• Exact Fractional Inference via Re-Parametrization & Interpolation between Tree-Re-Weighted- and Belief Propagation- Algorithms [0.4527270266697462]
  We show how to express $Z$ as a product, $forall lambda: Z=Z(lambda)tilde Z(lambda)$, where the multiplicative correction, $tilde Z(lambda)$, is an expectation over a node-independent probability distribution. We also discuss the applicability of this approach to the problem of image de-noising.
  arXiv  Detail & Related papers  (2023-01-25T00:50:28Z)
• The Approximate Degree of DNF and CNF Formulas [95.94432031144716]
  For every $delta>0,$ we construct CNF and formulas of size with approximate degree $Omega(n1-delta),$ essentially matching the trivial upper bound of $n. We show that for every $delta>0$, these models require $Omega(n1-delta)$, $Omega(n/4kk2)1-delta$, and $Omega(n/4kk2)1-delta$, respectively.
  arXiv  Detail & Related papers  (2022-09-04T10:01:39Z)
• Thermalization in many-fermion quantum systems with one- plus random $k$-body interactions [2.28438857884398]
  We study thermalization in finite many-fermion systems with random $k$-body interactions in presence of a mean-field. For higher body rank of interaction $k$, system thermalizes faster.
  arXiv  Detail & Related papers  (2022-06-21T15:24:40Z)
• Spectral properties of sample covariance matrices arising from random matrices with independent non identically distributed columns [50.053491972003656]
  It was previously shown that the functionals $texttr(AR(z))$, for $R(z) = (frac1nXXT- zI_p)-1$ and $Ain mathcal M_p$ deterministic, have a standard deviation of order $O(|A|_* / sqrt n)$. Here, we show that $|mathbb E[R(z)] - tilde R(z)|_F
  arXiv  Detail & Related papers  (2021-09-06T14:21:43Z)
• Power-like potentials: from the Bohr-Sommerfeld energies to exact ones [77.34726150561087]
  Bohr-Sommerfeld Energies (BSE) extracted explicitly from the Bohr-Sommerfeld quantization condition are compared with the exact energies. For physically important cases $m=1,4,6$ for the $100$th excited state BSE coincide with exact ones in 5-6 figures.
  arXiv  Detail & Related papers  (2021-07-31T21:37:50Z)
• Anharmonic oscillator: a solution [77.34726150561087]
  The dynamics in $x$-space and in $(gx)-space corresponds to the same energy spectrum with effective coupling constant $hbar g2$. A 2-classical generalization leads to a uniform approximation of the wavefunction in $x$-space with unprecedented accuracy.
  arXiv  Detail & Related papers  (2020-11-29T22:13:08Z)
• Wavefunction structure in quantum many-fermion systems with $k$-body interactions: conditional $q$-normal form of strength functions [0.40357531559576965]
  For finite quantum many-particle systems modeled with say $m$ fermions in $N$ single particle states, the wavefunction structure is studied using random matrix theory. It is shown that the first four moments of $F_kappa(E)$ are essentially same as that of the conditional $q$-normal distribution.
  arXiv  Detail & Related papers  (2020-11-09T11:53:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.