Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory

• URL: http://arxiv.org/abs/2604.07401v1
• Date: Wed, 08 Apr 2026 09:21:22 GMT
• Title: Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory
• Abstract summary: We study the memory capacity of modern Hopfield networks with continuous states under geometric constraints.<n>We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $p = eN$.
• Score: 0.9940728137241215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We study the thermodynamic memory capacity of modern Hopfield networks (Dense Associative Memory models) with continuous states under geometric constraints, extending classical analyses of pairwise associative memory. We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $p = e^{αN}$, comparing Gaussian (LSE) and Epanechnikov (LSR) kernels. For continuous neurons on an $N$-sphere, the geometric entropy depends solely on the spherical geometry, not the kernel. In the sharp-kernel regime, the maximum theoretical capacity $α= 0.5$ is achieved at zero temperature; below this threshold, a critical line separates retrieval from a spin-glass phase. The two kernels differ qualitatively in their phase boundary structure: for LSE, the retrieval region extends to arbitrarily high temperatures as $α\to 0$, but interference from spurious patterns is always present. For LSR, the finite support introduces a threshold $α_{\text{th}}$ below which no spurious patterns contribute to the noise floor, producing a qualitatively different retrieval regime in this sub-threshold region. These results advance the theory of high-capacity associative memory and clarify fundamental limits of retrieval robustness in modern attention-like memory architectures.

Related papers

• Phases in a class of associative memories via hidden neurons [5.577596968571274]
  Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system.<n>We study the bipartite architecture of Krotov and Hopfield, which we call the class $H$.
  arXiv  Detail & Related papers  (2026-09-10T01:50:32Z)
• Memory as an Energy Landscape---Hopfield [0.0]
  This chapter reconstructs the Hopfield network as a physical theory of memory.<n>It begins with the problem as it stood before 1982--and isolates what Hopfield's synthesis added.<n>It treats the Hopfield network as an effective theory whose symmetry, locality, and point-neuron assumptions delimit its biological reach.
  arXiv  Detail & Related papers  (2026-09-02T07:00:01Z)
• On the robustness of noisy solutions in non-convex neural networks [42.548424354574585]
  We show that frozen one-step replica-symmetry-breaking solution densities survive at any finite temperature.<n>We show that dense, algorithmically accessible regions of finite-energy persist beyond $_rm OGP$, up to a threshold $_rm OGP()$ that grows with the allowed error.<n>In the teacher-student setting, we show that these wide, finite-energy regions still retain good generalization.
  arXiv  Detail & Related papers  (2026-07-29T14:57:26Z)
• Free energy landscape of Dense Associative Memory [2.636047873931302]
  We solve and obtain a general expression for the free energy functional for a broad class of associative memories.<n>For a finite number of patterns, we derive the temperature-dependent free energy functional for dense associative memories.<n>Our framework reveals how memory retrieval depends on the initial state in higher-order dense networks.
  arXiv  Detail & Related papers  (2026-07-21T15:30:53Z)
• Anomalous Mixed-State Floquet Topology in One-Dimensional Open Quantum Systems [41.99844472131922]
  We investigate the non-equilibrium topology of a periodically driven, dissipative Su-Schrieffer-Heeger chain.<n>We show that the steady state can be characterised by a Hermitian purity spectrum.<n>We identify a pair of topological invariants $(0_mathrmEGP, _mathrmEGP)$, revealing a structure consistent with a $mathbbZtimesmathbbZ$ classification known from isolated Floquet SSH systems.
  arXiv  Detail & Related papers  (2026-04-28T05:54:40Z)
• Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels [0.16921396880325779]
  We map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the $N$-sphere with an exponential number of stored patterns.<n>We find that the LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature.
  arXiv  Detail & Related papers  (2026-03-07T17:06:08Z)
• Lowering the temperature of two-dimensional fermionic tensor networks with cluster expansions [0.0]
  Suzuki-Trotter decomposition is the main technique for representing a Hamiltonian composed of strictly short-ranged interactions.<n>We extend the cluster expansion to two-dimensional fermionic systems, and employ it to construct projected entangled-pair operator (PEPO) approximations of Gibbs states.
  arXiv  Detail & Related papers  (2026-02-25T16:58:58Z)
• When Less is More: Approximating the Quantum Geometric Tensor with Block Structures [43.39839287869652]
  We introduce a block-diagonal quantum geometric tensor that partitions the metric by network layers, analogous to block-structured Fisher methods such as K-FAC.<n> Experiments on Heisenberg and frustrated $J_1$-$J$ models show faster convergence, lower energy, and improved stability.
  arXiv  Detail & Related papers  (2025-10-09T16:44:30Z)
• Higher-Order Kuramoto Oscillator Network for Dense Associative Memory [0.0]
  We show that higher-order couplings achieve superlinear scaling of memory capacity with system size.<n>These results bridge the Kuramoto synchronization with modern Hopfield memories.
  arXiv  Detail & Related papers  (2025-07-29T16:35:52Z)
• High-temperature partition functions and classical simulability of long-range quantum systems [0.0]
  We study fundamental properties of long-range spin systems in thermal equilibrium.<n>Our main result is a proof of analiticity of their partition functions at high temperatures.
  arXiv  Detail & Related papers  (2025-04-29T16:17:45Z)
• KPZ scaling from the Krylov space [83.88591755871734]
  Recently, a superdiffusion exhibiting the Kardar-Parisi-Zhang scaling in late-time correlators and autocorrelators has been reported. Inspired by these results, we explore the KPZ scaling in correlation functions using their realization in the Krylov operator basis.
  arXiv  Detail & Related papers  (2024-06-04T20:57:59Z)
• Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks [54.177130905659155]
  Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks. In this paper, we study a suitable function space for over- parameterized two-layer neural networks with bounded norms.
  arXiv  Detail & Related papers  (2024-04-29T15:04:07Z)
• Contrasting pseudo-criticality in the classical two-dimensional Heisenberg and $\mathrm{RP}^2$ models: zero-temperature phase transition versus finite-temperature crossover [0.0]
  We compare the two-dimensional classical Heisenberg and $mathrmRP2$ models. For the Heisenberg model, we find no signs of a finite-temperature phase transition. For the $mathrmRP2$ model, we observe an abrupt onset of scaling behaviour.
  arXiv  Detail & Related papers  (2022-02-15T17:35:15Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Dimension-free convergence rates for gradient Langevin dynamics in RKHS [47.198067414691174]
  Gradient Langevin dynamics (GLD) and SGLD have attracted considerable attention lately. We provide a convergence analysis GLD and SGLD when the space is an infinite dimensional space.
  arXiv  Detail & Related papers  (2020-02-29T17:14:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.