Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels

• URL: http://arxiv.org/abs/2603.13350v1
• Date: Sat, 07 Mar 2026 17:06:08 GMT
• Title: Thermal Robustness of Retrieval in Dense Associative Memories: LSE vs LSR Kernels
• Abstract summary: We map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the $N$-sphere with an exponential number of stored patterns.<n>We find that the LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature.
• Score: 0.16921396880325779
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding whether retrieval in dense associative memories survives thermal noise is essential for bridging zero-temperature capacity proofs with the finite-temperature conditions of practical inference and biological computation. We use Monte Carlo simulations to map the retrieval phase boundary of two continuous dense associative memories (DAMs) on the $N$-sphere with an exponential number of stored patterns $M = e^{αN}$: a log-sum-exp (LSE) kernel and a log-sum-ReLU (LSR) kernel. Both kernels share the zero-temperature critical load $α_c(0)=0.5$, but their finite-temperature behavior differs markedly. The LSE kernel sustains retrieval at arbitrarily high temperatures for sufficiently low load, whereas the LSR kernel exhibits a finite support threshold below which retrieval is perfect at any temperature; for typical sharpness values this threshold approaches $α_c$, making retrieval nearly perfect across the entire load range. We also compare the measured equilibrium alignment with analytical Boltzmann predictions within the retrieval basin.

Related papers

• Fast PAC Global Optimization via Restarted Langevin: Exploration, Exploitation, and Degenerate Cooling [7.018547803286913]
  Algorithm design and analysis are in continuous time.<n>Global exploration is decoupled from exploitation.<n>Experiments on the six camel and Rastrigin objectives illustrate the benefits of warmer exploration.
  arXiv  Detail & Related papers  (2026-09-05T17:36:18Z)
• On the robustness of noisy solutions in non-convex neural networks [42.548424354574585]
  We show that frozen one-step replica-symmetry-breaking solution densities survive at any finite temperature.<n>We show that dense, algorithmically accessible regions of finite-energy persist beyond $_rm OGP$, up to a threshold $_rm OGP()$ that grows with the allowed error.<n>In the teacher-student setting, we show that these wide, finite-energy regions still retain good generalization.
  arXiv  Detail & Related papers  (2026-07-29T14:57:26Z)
• How Neural Reward Models Learn Features for Policy Optimization: A Single-Index Analysis [53.063298916923976]
  We study the feedback in a Gaussian single-index model with $r*(x) = *(langle *, xrangle)$ and $x sim N(0, I_d)$.<n>We analyze a two-stage neural reward model that first learns the hidden direction $*$ from reward-weighted samples and then fits the readout layer by weighted ridge regression.
  arXiv  Detail & Related papers  (2026-05-23T22:00:38Z)
• Algorithmic Analysis of Dense Associative Memory: Finite-Size Guarantees and Adversarial Robustness [0.0]
  We study Dense Associative Memory (DAM) generalizes Hopfield networks through higher-order interactions.<n>We derive capacity guarantees that scale as $(Nn-1)$ up to polylogarithmic factors in the worst case.<n>We show that DAM retrieval dynamics admit a potential-game interpretation that ensures convergence to pure Nash equilibria.
  arXiv  Detail & Related papers  (2026-04-14T14:38:46Z)
• Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory [0.9940728137241215]
  We study the memory capacity of modern Hopfield networks with continuous states under geometric constraints.<n>We derive thermodynamic phase boundaries for Dense Associative Memory networks with exponential capacity $p = eN$.
  arXiv  Detail & Related papers  (2026-04-08T09:21:22Z)
• Assessing the Limits of Graph Neural Networks for Vapor-Liquid Equilibrium Prediction: A Cryogenic Mixture Case Study [3.765010805872486]
  This study asks whether a structure-aware graph neural network (GNN; DimeNet++) trained on GERG-2008/CoolProp data can act as a practical surrogate for an equation of state (EoS)<n>We generate a ternary dataset over 90-200 K and pressures to 100 bar, curate it with a 15% density filter (reducing 5,200 states to 1,516), and pair each state with a lightweight molecular-dynamics snapshot to supply structural features.<n>We conclude that, as configured, the surrogate in this study is not equilibrium-ready for VLE and offers no runtime benefit.
  arXiv  Detail & Related papers  (2025-09-10T16:10:58Z)
• Dense Associative Memory with Epanechnikov Energy [31.882557303410483]
  We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR)<n>Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel.<n> Empirical results show that LSR energy has significantly more local minima (memories) that have comparable log-likelihood to LSE-based models.
  arXiv  Detail & Related papers  (2025-06-12T15:16:31Z)
• Scaling Relations of Spectrum Form Factor and Krylov Complexity at Finite Temperature [2.25304964649011]
  We extend the analysis to include the finite temperature effects on the Krylov complexity and SFF. Our work deepens the understanding of the finite temperature effects on Krylov complexity, SFF, and the connection between ergodicity and operator growth.
  arXiv  Detail & Related papers  (2024-01-19T05:28:10Z)
• Detection-Recovery Gap for Planted Dense Cycles [72.4451045270967]
  We consider a model where a dense cycle with expected bandwidth $n tau$ and edge density $p$ is planted in an ErdHos-R'enyi graph $G(n,q)$. We characterize the computational thresholds for the associated detection and recovery problems for the class of low-degree algorithms.
  arXiv  Detail & Related papers  (2023-02-13T22:51:07Z)
• Mean-Square Analysis with An Application to Optimal Dimension Dependence of Langevin Monte Carlo [60.785586069299356]
  This work provides a general framework for the non-asymotic analysis of sampling error in 2-Wasserstein distance. Our theoretical analysis is further validated by numerical experiments.
  arXiv  Detail & Related papers  (2021-09-08T18:00:05Z)
• Uhlmann Fidelity and Fidelity Susceptibility for Integrable Spin Chains at Finite Temperature: Exact Results [68.8204255655161]
  We show that the proper inclusion of the odd parity subspace leads to the enhancement of maximal fidelity susceptibility in the intermediate range of temperatures. The correct low-temperature behavior is captured by an approximation involving the two lowest many-body energy eigenstates.
  arXiv  Detail & Related papers  (2021-05-11T14:08:02Z)
• Out-of-time-order correlations and the fine structure of eigenstate thermalisation [58.720142291102135]
  Out-of-time-orderors (OTOCs) have become established as a tool to characterise quantum information dynamics and thermalisation. We show explicitly that the OTOC is indeed a precise tool to explore the fine details of the Eigenstate Thermalisation Hypothesis (ETH) We provide an estimation of the finite-size scaling of $omega_textrmGOE$ for the general class of observables composed of sums of local operators in the infinite-temperature regime.
  arXiv  Detail & Related papers  (2021-03-01T17:51:46Z)
• Faster Convergence of Stochastic Gradient Langevin Dynamics for Non-Log-Concave Sampling [110.88857917726276]
  We provide a new convergence analysis of gradient Langevin dynamics (SGLD) for sampling from a class of distributions that can be non-log-concave. At the core of our approach is a novel conductance analysis of SGLD using an auxiliary time-reversible Markov Chain.
  arXiv  Detail & Related papers  (2020-10-19T15:23:18Z)
• Quantum Monte Carlo study of strongly interacting bosonic one-dimensional systems in periodic potentials [0.0]
  We present Monte Carlo calculations of a one-dimensional Bose system with realistic interparticle interactions in a periodic external potential. Our main aim is to test the predictions of the Luttinger liquid (LL) theory, in particular with respect to the super-Mott insulator transition at both zero and finite temperatures.
  arXiv  Detail & Related papers  (2020-01-20T16:26:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.