Memory as an Energy Landscape---Hopfield

• URL: http://arxiv.org/abs/2609.02195v1
• Date: Wed, 02 Sep 2026 07:00:01 GMT
• Title: Memory as an Energy Landscape---Hopfield
• Abstract summary: This chapter reconstructs the Hopfield network as a physical theory of memory.<n>It begins with the problem as it stood before 1982--and isolates what Hopfield's synthesis added.<n>It treats the Hopfield network as an effective theory whose symmetry, locality, and point-neuron assumptions delimit its biological reach.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This chapter reconstructs the Hopfield network as a physical theory of memory rather than merely an early neural-network algorithm. It begins with the problem as it stood before 1982-threshold logic, Hebbian association, correlation memories, and recurrent binary networks-and isolates what Hopfield's synthesis added: a dynamical definition of content-addressable memory, a symmetric recurrent architecture with a Lyapunov function, a Hebbian embedding of patterns in its couplings, and a physical account of basins, robustness, and graceful degradation. The binary and graded-response energy functions are derived in full, together with the signal-crosstalk decomposition governing pattern stability, the mean-field theory of retrieval at extensive load, and the zero-temperature retrieval spinodal at (alpha 0.138) established by Amit, Gutfreund, and Sompolinsky. The energy-based program is then followed through analog optimization networks, polynomial dense associative memories, exponential interactions, and modern continuous Hopfield updates, including the precise conditions under which the update becomes scaled dot-product attention. Throughout, capacity claims are tied to their disorder ensemble, scaling limit, and success criterion, showing why numerically different storage limits need not conflict. A closing assessment distinguishes established results from surviving principles, assumption-bound limitations, and open problems, treating the Hopfield network as an effective theory whose symmetry, locality, and point-neuron assumptions delimit its biological reach. Fixed-seed numerical experiments expose the mechanisms discussed but do not substitute for analytical results.

Related papers

• Phases in a class of associative memories via hidden neurons [5.577596968571274]
  Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system.<n>We study the bipartite architecture of Krotov and Hopfield, which we call the class $H$.
  arXiv  Detail & Related papers  (2026-09-10T01:50:32Z)
• Geometric and dynamical analysis of attractor boundaries and storage limits in kernel Hopfield networks [0.0]
  This paper investigates the global geometry of attractor basins and the mechanisms governing the storage limit in KLR-trained Hopfield networks.<n>Experiments show that the network achieves a storage capacity for random sequences up to $P/N approx 16$, while maintaining stable retrieval for structured data at effective loads near $P/N approx 20$.
  arXiv  Detail & Related papers  (2026-05-01T03:04:44Z)
• Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory [0.9940728137241215]
  We study the memory capacity of modern Hopfield networks with continuous states under geometric constraints.<n>We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $p = eN$.
  arXiv  Detail & Related papers  (2026-04-08T09:21:22Z)
• On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking [49.1352577985191]
  We present a comprehensive analysis of how two-layer neural networks learn features to solve the modular addition task.<n>Our work provides a full mechanistic interpretation of the learned model and a theoretical explanation of its training dynamics.
  arXiv  Detail & Related papers  (2026-02-18T20:25:13Z)
• Spin Glass Dynamics on Complex Hardware Topologies: A Bond-Correlated Percolation Approach [41.99844472131922]
  We study the non-exponential relaxation behavior of spin glasses on network architectures relevant to quantum annealing hardware.<n>This framework enables a more comprehensive characterization of spin-glass energy landscapes.
  arXiv  Detail & Related papers  (2025-10-28T18:16:59Z)
• Contraction, Criticality, and Capacity: A Dynamical-Systems Perspective on Echo-State Networks [13.857230672081489]
  We present a unified, dynamical-systems treatment that weaves together functional analysis, random attractor theory and recent neuroscientific findings.<n>First, we prove that the Echo-State Property (wash-out of initial conditions) together with global Lipschitz dynamics necessarily yields the Fading-Memory Property.<n>Second, employing a Stone-Weierstrass strategy we give a streamlined proof that ESNs with nonlinear reservoirs and linear read-outs are dense in the Banach space of causal, time-in fading-memory filters.<n>Third, we quantify computational resources via memory-capacity spectrum, show how
  arXiv  Detail & Related papers  (2025-07-24T14:41:18Z)
• Latent Structured Hopfield Network for Semantic Association and Retrieval [52.634915010996835]
  Episodic memory enables humans to recall past experiences by associating semantic elements such as objects, locations, and time into coherent event representations.<n>We propose the Latent Structured Hopfield Network (LSHN), a framework that integrates continuous Hopfield attractor dynamics into an autoencoder architecture.<n>Unlike traditional Hopfield networks, our model is trained end-to-end with gradient descent, achieving scalable and robust memory retrieval.
  arXiv  Detail & Related papers  (2025-06-02T04:24:36Z)
• KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches [43.872190335490515]
  This paper introduces KITINet, a novel architecture that reinterprets feature propagation through the lens of non-equilibrium particle dynamics.<n>At its core, we propose a residual module that models update as the evolution of a particle system.<n>This formulation mimics particle collisions and energy exchange, enabling adaptive feature refinement via physics-informed interactions.
  arXiv  Detail & Related papers  (2025-05-23T13:58:29Z)
• Allostatic Control of Persistent States in Spiking Neural Networks for perception and computation [79.16635054977068]
  We introduce a novel model for updating perceptual beliefs about the environment by extending the concept of Allostasis to the control of internal representations.<n>In this paper, we focus on an application in numerical cognition, where a bump of activity in an attractor network is used as a spatial numerical representation.
  arXiv  Detail & Related papers  (2025-03-20T12:28:08Z)
• Explosive neural networks via higher-order interactions in curved statistical manifolds [43.496401697112695]
  We introduce curved neural networks as a class of prototypical models with a limited number of parameters.<n>We show that these curved neural networks implement a self-regulating process that can accelerate memory retrieval.<n>We analytically explore their memory-retrieval capacity using the replica trick near ferromagnetic and spin-glass phase boundaries.
  arXiv  Detail & Related papers  (2024-08-05T09:10:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.