Hierarchical Maximum Entropy via the Renormalization Group

• URL: http://arxiv.org/abs/2509.01424v1
• Date: Mon, 01 Sep 2025 12:30:23 GMT
• Title: Hierarchical Maximum Entropy via the Renormalization Group
• Abstract summary: Hierarchical structures, which include multiple levels, are prevalent in statistical and machine-learning models as well as physical systems.<n>We introduce the framework of "hierarchical maximum entropy" to address these multilevel models.<n>This work connects ideas from probability theory, information theory, and statistical mechanics.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hierarchical structures, which include multiple levels, are prevalent in statistical and machine-learning models as well as physical systems. Extending the foundational result that the maximum entropy distribution under mean constraints is given by the exponential Gibbs-Boltzmann form, we introduce the framework of "hierarchical maximum entropy" to address these multilevel models. We demonstrate that Pareto optimal distributions, which maximize entropies across all levels of hierarchical transformations, can be obtained via renormalization-group procedures from theoretical physics. This is achieved by formulating multilevel extensions of the Gibbs variational principle and the Donsker-Varadhan variational representation of entropy. Moreover, we explore settings with hierarchical invariances that significantly simplify the renormalization-group procedures, enhancing computational efficiency: quadratic modular loss functions, logarithmic loss functions, and nearest-neighbor loss functions. This is accomplished through the introduction of the concept of parameter flows, which serves as an analog to renormalization flows in renormalization group theory. This work connects ideas from probability theory, information theory, and statistical mechanics.

Related papers

• Beyond Perturbation Theory: A Resolvent-Based Framework for Strongly Correlated Many-Body Systems [1.4824891788575416]
  Traditional perturbation theory fails in many-body systems with exponentially small energy gaps and strong interactions.<n>This work presents an alternative methodological framework built on two core principles.<n>It is applicable to the quantitative analysis of global properties in nonintegrable many-body systems.
  arXiv  Detail & Related papers  (2026-04-01T08:11:58Z)
• Group Entropies and Mirror Duality: A Class of Flexible Mirror Descent Updates for Machine Learning [8.605564917115663]
  We introduce a comprehensive theoretical and algorithmic framework that bridges formal group theory and group entropies with modern machine learning.<n>Our approach exploits the rich structure of group entropies, which are generalized entropic functionals governed by group composition laws.
  arXiv  Detail & Related papers  (2026-03-09T17:31:03Z)
• Conjugate Learning Theory: Uncovering the Mechanisms of Trainability and Generalization in Deep Neural Networks [0.0]
  We develop a conjugate learning theoretical framework based on convex conjugate duality to characterize this learnability property.<n>We demonstrate that training deep neural networks (DNNs) with mini-batch descent (SGD) achieves global optima of empirical risk.<n>We derive deterministic and probabilistic bounds on generalization error based on conditional generalized entropy measures.
  arXiv  Detail & Related papers  (2026-02-18T04:26:55Z)
• Rethinking Diffusion Models with Symmetries through Canonicalization with Applications to Molecular Graph Generation [56.361076943802594]
  CanonFlow achieves state-of-the-art performance on the challenging GEOM-DRUG dataset, and the advantage remains large in few-step generation.
  arXiv  Detail & Related papers  (2026-02-16T18:58:55Z)
• Random-Matrix-Induced Simplicity Bias in Over-parameterized Variational Quantum Circuits [72.0643009153473]
  We show that expressive variational ansatze enter a Haar-like universality class in which both observable expectation values and parameter gradients concentrate exponentially with system size.<n>As a consequence, the hypothesis class induced by such circuits collapses with high probability to a narrow family of near-constant functions.<n>We further show that this collapse is not unavoidable: tensor-structured VQCs, including tensor-network-based and tensor-hypernetwork parameterizations, lie outside the Haar-like universality class.
  arXiv  Detail & Related papers  (2026-01-05T08:04:33Z)
• Free Cumulants and Full Eigenstate Thermalization from Boundary Scrambling [0.0]
  We introduce a solvable many-body quantum model, which we term boundary scrambling, for which the full dynamics of higher-order OTOCs is analytically tractable.<n>We obtain exact expressions for (higher-order) correlations between matrix elements and show these to be stable away from the solvable point.<n>These results provide insight into the emergence of random-matrix behavior from structured Floquet dynamics and show how techniques from free probability can be applied in the construction of exactly-solvable many-body models.
  arXiv  Detail & Related papers  (2025-09-09T18:01:45Z)
• Loss-Complexity Landscape and Model Structure Functions [56.01537787608726]
  We develop a framework for dualizing the Kolmogorov structure function $h_x(alpha)$.<n>We establish a mathematical analogy between information-theoretic constructs and statistical mechanics.<n>We explicitly prove the Legendre-Fenchel duality between the structure function and free energy.
  arXiv  Detail & Related papers  (2025-07-17T21:31:45Z)
• Designing a Linearized Potential Function in Neural Network Optimization Using Csiszár Type of Tsallis Entropy [0.0]
  In this paper, we establish a framework that utilizes a linearized potential function via Csisz'ar type of Tsallis entropy. We show that our new framework enable us to derive an exponential convergence result.
  arXiv  Detail & Related papers  (2024-11-06T02:12:41Z)
• Entanglement and the density matrix renormalisation group in the generalised Landau paradigm [0.0]
  We leverage the interplay between gapped phases and dualities of symmetric one-dimensional quantum lattice models. For every phase in the phase diagram, the dual representation of the ground state that breaks all symmetries minimises both the entanglement entropy and the required number of variational parameters. Our work testifies to the usefulness of generalised non-invertible symmetries and their formal category theoretic description for the nuts and bolts simulation of strongly correlated systems.
  arXiv  Detail & Related papers  (2024-08-12T17:51:00Z)
• DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion [66.21290235237808]
  We introduce an energy constrained diffusion model which encodes a batch of instances from a dataset into evolutionary states. We provide rigorous theory that implies closed-form optimal estimates for the pairwise diffusion strength among arbitrary instance pairs. Experiments highlight the wide applicability of our model as a general-purpose encoder backbone with superior performance in various tasks.
  arXiv  Detail & Related papers  (2023-01-23T15:18:54Z)
• Path-integral approaches to strongly-coupled quantum many-body systems [0.0]
  The core of this thesis is the path-integral formulation of quantum field theory.<n>The efficiency of theoretical approaches in the treatment of finite-size quantum systems can be studied.
  arXiv  Detail & Related papers  (2022-10-29T19:41:03Z)
• Renormalization Group Flow as Optimal Transport [0.0]
  Polchinski's equation for exact renormalization group flow is equivalent to the optimal transport gradient flow of a field-theoretic relative entropy. A striking consequence is that a regularization of the relative entropy is in fact an RG monotone.
  arXiv  Detail & Related papers  (2022-02-23T19:02:01Z)
• Joint Network Topology Inference via Structured Fusion Regularization [70.30364652829164]
  Joint network topology inference represents a canonical problem of learning multiple graph Laplacian matrices from heterogeneous graph signals. We propose a general graph estimator based on a novel structured fusion regularization. We show that the proposed graph estimator enjoys both high computational efficiency and rigorous theoretical guarantee.
  arXiv  Detail & Related papers  (2021-03-05T04:42:32Z)
• Action Redundancy in Reinforcement Learning [54.291331971813364]
  We show that transition entropy can be described by two terms; namely, model-dependent transition entropy and action redundancy. Our results suggest that action redundancy is a fundamental problem in reinforcement learning.
  arXiv  Detail & Related papers  (2021-02-22T19:47:26Z)
• Maximum Multiscale Entropy and Neural Network Regularization [28.00914218615924]
  A well-known result shows that the maximum entropy distribution under a mean constraint has an exponential form called the Gibbs-Boltzmann distribution. This paper investigates a generalization of these results to a multiscale setting.
  arXiv  Detail & Related papers  (2020-06-25T17:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.