Phases in a class of associative memories via hidden neurons

• URL: http://arxiv.org/abs/2609.10976v1
• Date: Thu, 10 Sep 2026 01:50:32 GMT
• Title: Phases in a class of associative memories via hidden neurons
• Abstract summary: Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system.<n>We study the bipartite architecture of Krotov and Hopfield, which we call the class $H$.
• Score: 5.577596968571274
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class $H$, whose model is fixed by a Lagrangian for each layer, taking the hidden neurons as the order parameter of retrieval. At polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible neurons, so their differences come from the visible entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes retrieval by quantized reassignments of attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at polynomial load and large-deviation at exponential load, and the class $H$ splits retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.

Related papers

• Toward a mechanistic understanding of inference in visual cortex and diffusion models [46.86106767256413]
  We describe a model of perceptual inference in primary visual cortex (V1)<n>We efficiently train these recurrent dynamics using a denoising score-matching objective and implicit differentiation.<n>This model exhibits exceptionally good denoising performance, restoring image features such as extended contours amid extreme visual ambiguity.
  arXiv  Detail & Related papers  (2026-07-17T07:11:57Z)
• Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory [0.9940728137241215]
  We study the memory capacity of modern Hopfield networks with continuous states under geometric constraints.<n>We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $p = eN$.
  arXiv  Detail & Related papers  (2026-04-08T09:21:22Z)
• Symmetry-protected topology and deconfined solitons in a multi-link $\mathbb{Z}_2$ gauge theory [45.88028371034407]
  We study a $mathbbZ$ lattice gauge theory defined on a multi-graph with links that can be visualized as great circles of a spherical shell.<n>We show that this leads to state-dependent tunneling amplitudes underlying a phenomenon analogous to the Peierls instability.<n>By performining a detailed analysis based on matrix product states, we prove that charge deconfinement emerges as a consequence of charge-fractionalization.
  arXiv  Detail & Related papers  (2026-03-02T22:59:25Z)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking [49.1352577985191]
  We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task.<n>Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics.
  arXiv  Detail & Related papers  (2026-02-18T20:25:13Z)
• Quantum dynamics in lattices in presence of bulk dephasing and a localized source [0.0]
  We study the dynamics of quantum systems subjected to a localized fermionic source in the presence of bulk dephasing.<n>Our detailed study reveals an interesting interplay between Hamiltonian dynamics and various environmentally induced mechanisms in open quantum systems.
  arXiv  Detail & Related papers  (2025-11-01T14:42:52Z)
• Provable Scaling Laws of Feature Emergence from Learning Dynamics of Grokking [44.614763110719274]
  We study the phenomenon of grokking, i.e., delayed generalization.<n>We propose a novel framework that captures three key stages for the grokking behavior of 2-layer nonlinear networks.<n>Our study sheds lights on roles played by hypers such as weight decay, learning rate and sample sizes in grokking.
  arXiv  Detail & Related papers  (2025-09-25T20:08:09Z)
• Exponential onset of scalable entanglement via twist-and-turn dynamics in XY models [41.94295877935867]
  We show that the so-called twist-and-turn" (TaT) dynamics can offer an important resource to reach scalable multipartite entanglement.<n>For dipolar interactions, the entanglement dynamics at intermediate times is completely at odds with thermalization.
  arXiv  Detail & Related papers  (2025-07-10T22:34:44Z)
• Quench dynamics in higher-dimensional Holstein models: Insights from Truncated Wigner Approaches [41.94295877935867]
  We study the melting of charge-density waves in a Holstein model after a sudden switch-on of the electronic hopping. A comparison with exact data obtained for a Holstein chain shows that a semiclassical treatment of both the electrons and phonons is required in order to correctly describe the phononic dynamics.
  arXiv  Detail & Related papers  (2023-12-19T16:14:01Z)
• Onset of scrambling as a dynamical transition in tunable-range quantum circuits [0.0]
  We identify a dynamical transition marking the onset of scrambling in quantum circuits with different levels of long-range connectivity. We show that as a function of the interaction range for circuits of different structures, the tripartite mutual information exhibits a scaling collapse. In addition to systems with conventional power-law interactions, we identify the same phenomenon in deterministic, sparse circuits.
  arXiv  Detail & Related papers  (2023-04-19T17:37:10Z)
• Thermodynamics of bidirectional associative memories [0.0]
  We investigate the equilibrium properties of bidirectional associative memories (BAMs) introduced by Kosko in 1988 as a generalization of the Hopfield model to a bipartite structure. We characterize the computational capabilities of a extension of this model in the thermodynamic limit.
  arXiv  Detail & Related papers  (2022-11-17T17:35:37Z)
• Continuous and time-discrete non-Markovian system-reservoir interactions: Dissipative coherent quantum feedback in Liouville space [62.997667081978825]
  We investigate a quantum system simultaneously exposed to two structured reservoirs. We employ a numerically exact quasi-2D tensor network combining both diagonal and off-diagonal system-reservoir interactions with a twofold memory for continuous and discrete retardation effects. As a possible example, we study the non-Markovian interplay between discrete photonic feedback and structured acoustic phononovian modes, resulting in emerging inter-reservoir correlations and long-living population trapping within an initially-excited two-level system.
  arXiv  Detail & Related papers  (2020-11-10T12:38:35Z)
• Memory kernel and divisibility of Gaussian Collisional Models [0.0]
  Memory effects in the dynamics of open systems have been the subject of significant interest in the last decades. We analyze two types of interactions, a beam-splitter implementing a partial SWAP and a two-mode squeezing, which entangles the ancillas and feeds excitations into the system. By analyzing the memory kernel and divisibility for these two representative scenarios, our results help to shed light on the intricate mechanisms behind memory effects in the quantum domain.
  arXiv  Detail & Related papers  (2020-08-03T10:28:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.