Free energy landscape of Dense Associative Memory

• URL: http://arxiv.org/abs/2607.19195v2
• Date: Wed, 22 Jul 2026 20:42:50 GMT
• Title: Free energy landscape of Dense Associative Memory
• Abstract summary: We solve and obtain a general expression for the free energy functional for a broad class of associative memories.<n>For a finite number of patterns, we derive the temperature-dependent free energy functional for dense associative memories.<n>Our framework reveals how memory retrieval depends on the initial state in higher-order dense networks.
• Score: 2.636047873931302
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Using large deviations theory, we solve and obtain a general expression for the free energy functional for a broad class of associative memories, including dense associative memories. We illustrate the method by reproducing classical results for the Hopfield model. For a finite number of patterns, we derive the temperature-dependent free energy functional for dense associative memories featuring polynomial interactions and Log-Sum-Exponential (LSE) activation. We also evaluate the disorder-averaged ground-state energy of these systems in the extensive limit. Our analytical framework reveals how memory retrieval depends on the initial state in higher-order dense networks, and gives the exact full-retrieval threshold for the LSE model. This method provides a systematic procedure for analyzing diverse, complex architectures in associative memory.

Related papers

• Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory [0.9940728137241215]
  We study the memory capacity of modern Hopfield networks with continuous states under geometric constraints.<n>We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $p = eN$.
  arXiv  Detail & Related papers  (2026-04-08T09:21:22Z)
• Social Hippocampus Memory Learning [54.381608978482184]
  SoHip is a memory-centric social machine learning framework.<n>It enables collaboration among heterogeneous agents via memory sharing rather than model sharing.<n>Experiments on two benchmark datasets with seven baselines demonstrate that SoHip consistently outperforms existing methods.
  arXiv  Detail & Related papers  (2026-03-26T16:31:00Z)
• A Dynamical Theory of Sequential Retrieval in Input-Driven Hopfield Networks [0.0]
  This work develops a theory of sequential reasoning in Hopfield networks.<n>We derive explicit conditions for self-sustained memory transitions, including gain thresholds, escape times, and collapse regimes.
  arXiv  Detail & Related papers  (2026-03-03T17:54:36Z)
• Improving Multi-step RAG with Hypergraph-based Memory for Long-Context Complex Relational Modeling [83.29209853451697]
  Multi-step retrieval-augmented generation (RAG) has become a widely adopted strategy for enhancing large language models (LLMs)<n>We introduce HGMem, a hypergraph-based memory mechanism that extends the concept of memory into a dynamic, expressive structure for complex reasoning and global understanding.<n>In our approach, memory is represented as a hypergraph whose hyperedges correspond to distinct memory units, enabling the progressive formation of higher-order interactions within memory.
  arXiv  Detail & Related papers  (2025-12-30T03:13:10Z)
• Memory in the Age of AI Agents [217.9368190980982]
  This work aims to provide an up-to-date landscape of current agent memory research.<n>We identify three dominant realizations of agent memory, namely token-level, parametric, and latent memory.<n>To support practical development, we compile a comprehensive summary of memory benchmarks and open-source frameworks.
  arXiv  Detail & Related papers  (2025-12-15T17:22:34Z)
• Higher-Order Kuramoto Oscillator Network for Dense Associative Memory [0.0]
  We show that higher-order couplings achieve superlinear scaling of memory capacity with system size.<n>These results bridge the Kuramoto synchronization with modern Hopfield memories.
  arXiv  Detail & Related papers  (2025-07-29T16:35:52Z)
• Dense Associative Memory with Epanechnikov Energy [31.882557303410483]
  We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR)<n>Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel.<n> Empirical results show that LSR energy has significantly more local minima (memories) that have comparable log-likelihood to LSE-based models.
  arXiv  Detail & Related papers  (2025-06-12T15:16:31Z)
• Latent Structured Hopfield Network for Semantic Association and Retrieval [52.634915010996835]
  Episodic memory enables humans to recall past experiences by associating semantic elements such as objects, locations, and time into coherent event representations.<n>We propose the Latent Structured Hopfield Network (LSHN), a framework that integrates continuous Hopfield attractor dynamics into an autoencoder architecture.<n>Unlike traditional Hopfield networks, our model is trained end-to-end with gradient descent, achieving scalable and robust memory retrieval.
  arXiv  Detail & Related papers  (2025-06-02T04:24:36Z)
• Quantifying Memory Utilization with Effective State-Size [73.52115209375343]
  We develop a measure of textitmemory utilization'<n>This metric is tailored to the fundamental class of systems with textitinput-invariant and textitinput-varying linear operators
  arXiv  Detail & Related papers  (2025-04-28T08:12:30Z)
• Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models [65.08133391009838]
  generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks. We introduce a novel perspective to describe DMs using the mathematical language of memory retrieval from the field of energy-based Associative Memories (AMs) We present a growing body of evidence that records DMs exhibiting empirical behavior we would expect from AMs, and conclude by discussing research opportunities that are revealed by understanding DMs as a form of energy-based memory.
  arXiv  Detail & Related papers  (2023-09-28T17:57:09Z)
• Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models [41.58529335439799]
  We propose a general framework for understanding the operation of memory networks as a sequence of three operations. We derive all these memory models as instances of our general framework with differing similarity and separation functions.
  arXiv  Detail & Related papers  (2022-02-09T16:48:06Z)
• Targeted free energy estimation via learned mappings [66.20146549150475]
  Free energy perturbation (FEP) was proposed by Zwanzig more than six decades ago as a method to estimate free energy differences. FEP suffers from a severe limitation: the requirement of sufficient overlap between distributions. One strategy to mitigate this problem, called Targeted Free Energy Perturbation, uses a high-dimensional mapping in configuration space to increase overlap.
  arXiv  Detail & Related papers  (2020-02-12T11:10:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.