Reasoning as Attractor Dynamics: Latent Memory Retrieval via Gibbs-Weighted Energy Minimization

• URL: http://arxiv.org/abs/2606.24543v1
• Date: Tue, 23 Jun 2026 13:07:39 GMT
• Title: Reasoning as Attractor Dynamics: Latent Memory Retrieval via Gibbs-Weighted Energy Minimization
• Abstract summary: Large Language Models (LLMs) are traditionally viewed as autoregressive generators.<n>In this work, we investigate the energy landscape of mathematical reasoning.<n>We introduce a retrieval mechanism based on a Gibbs measure of the trajectory's spectral entropy.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Large Language Models (LLMs) are traditionally viewed as autoregressive generators. However, from the perspective of collective computation, they function as high-dimensional Dense Associative Memories that store complex reasoning patterns as latent attractors. In this work, we investigate the energy landscape of mathematical reasoning. We posit that correct reasoning chains correspond to deep, wide attractor basins ("flat minima") in the model's output distribution, whereas hallucinations manifest as sharp, unstable local minima. To exploit this geometry, we introduce a retrieval mechanism based on a Gibbs measure of the trajectory's spectral entropy. By sampling multiple reasoning paths and weighting them by their inverse energy ($P \propto e^{-βE}$), we approximate the equilibrium distribution of the associative memory, effectively ``relaxing'' the system into a robust solution. Empirically, this physics-inspired mechanism improves Microsoft Phi-3.5 performance on GSM8K by 5.38\% (84.7\% $\to$ 90.1\%), demonstrating that inference is better modeled as a dynamic settling process into an attractor basin rather than greedy next-token prediction.

Related papers

• Non-equilibrium thermodynamics of collapse models in the strongly non-Gaussian regime [0.0]
  We show that a dissipative Disi-Penrose (dDP) model induces complex, non-Gaussian phase-space dynamics.<n>We rigorously establish the thermodynamic consistency of this friction mechanism across both weakly and strongly non-Gaussian regimes.<n>Our analysis reveals that the system subjected to the dDP mechanism does not thermalize, but rather settles into a non-equilibrium steady-state.
  arXiv  Detail & Related papers  (2026-06-04T15:04:07Z)
• Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics [3.251449696624754]
  We propose a dense associative memory for empirical measures.<n>Stored patterns and queries are finitely supported probability measures.<n>We derive retrieval dynamics as a spherical Hellinger Kantorovich gradient flow.
  arXiv  Detail & Related papers  (2026-03-21T05:25:15Z)
• Thermodynamic Isomorphism of Transformers: A Lagrangian Approach to Attention Dynamics [0.0]
  We show that within the Shannon--Boltzmann entropy framework, the Softmax function arises as a stationary solution minimizing a Helmholtz free energy functional.<n> Extending this mapping to macroscopic observables, we define an effective specific heat associated with fluctuations of the attention energy landscape.<n>Our framework provides a unified statistical-mechanical perspective on attention scaling, training dynamics, and positional encoding.
  arXiv  Detail & Related papers  (2026-02-09T02:42:36Z)
• Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models [55.908141398092646]
  Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs.<n>We study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in the smaller models.<n>We show that small models such as $textttGPT2$ and $textttQwen3-0.6B$ exhibit severe condensation, whereas the larger models such as $textttGPT2-xl$ and $textttQwen3-32B
  arXiv  Detail & Related papers  (2026-01-30T16:07:03Z)
• Latent Object Permanence: Topological Phase Transitions, Free-Energy Principles, and Renormalization Group Flows in Deep Transformer Manifolds [0.5729426778193398]
  We study the emergence of multi-step reasoning in deep Transformer language models through a geometric and statistical-physics lens.<n>We formalize the forward pass as a discrete coarse-graining map and relate the appearance of stable "concept basins" to fixed points of this renormalization-like dynamics.<n>The resulting low-entropy regime is characterized by a spectral tail collapse and by the formation of transient, reusable object-like structures in representation space.
  arXiv  Detail & Related papers  (2026-01-16T23:11:02Z)
• Digital Metabolism: Decoupling Logic from Facts via Regenerative Unlearning -- Towards a Pure Neural Logic Core [4.073707521515039]
  "Digital metabolism" is a hypothesis suggesting that targeted forgetting is necessary for distilling a pure neural logic core.<n>We introduce the Regenerative Logic-Core Protocol (RLCP), a dual-stream training framework that renders specific factual dependencies linearly undecodable.<n> Empirical analysis on GSM8K reveals that the "metabolized" model spontaneously adopts Symbolic chain-of-thought scaffolding.
  arXiv  Detail & Related papers  (2026-01-15T19:21:16Z)
• Bayesian Circular Regression with von Mises Quasi-Processes [57.88921637944379]
  In this work we explore a family of expressive and interpretable distributions over circle-valued random functions.<n>For posterior inference, we introduce a new Stratonovich-like augmentation that lends itself to fast Gibbs sampling.<n>We present experiments applying this model to the prediction of wind directions and the percentage of the running gait cycle as a function of joint angles.
  arXiv  Detail & Related papers  (2024-06-19T01:57:21Z)
• Hamiltonian Mechanics of Feature Learning: Bottleneck Structure in Leaky ResNets [58.460298576330835]
  We study Leaky ResNets, which interpolate between ResNets and Fully-Connected nets depending on an 'effective depth'<n>We leverage this intuition to explain the emergence of a bottleneck structure, as observed in previous work.
  arXiv  Detail & Related papers  (2024-05-27T18:15:05Z)
• Modeling the space-time correlation of pulsed twin beams [68.8204255655161]
  Entangled twin-beams generated by parametric down-conversion are among the favorite sources for imaging-oriented applications. We propose a semi-analytic model which aims to bridge the gap between time-consuming numerical simulations and the unrealistic plane-wave pump theory.
  arXiv  Detail & Related papers  (2023-01-18T11:29:49Z)
• Out-of-time-order correlations and the fine structure of eigenstate thermalisation [58.720142291102135]
  Out-of-time-orderors (OTOCs) have become established as a tool to characterise quantum information dynamics and thermalisation. We show explicitly that the OTOC is indeed a precise tool to explore the fine details of the Eigenstate Thermalisation Hypothesis (ETH) We provide an estimation of the finite-size scaling of $omega_textrmGOE$ for the general class of observables composed of sums of local operators in the infinite-temperature regime.
  arXiv  Detail & Related papers  (2021-03-01T17:51:46Z)
• Probing eigenstate thermalization in quantum simulators via fluctuation-dissipation relations [77.34726150561087]
  The eigenstate thermalization hypothesis (ETH) offers a universal mechanism for the approach to equilibrium of closed quantum many-body systems. Here, we propose a theory-independent route to probe the full ETH in quantum simulators by observing the emergence of fluctuation-dissipation relations. Our work presents a theory-independent way to characterize thermalization in quantum simulators and paves the way to quantum simulate condensed matter pump-probe experiments.
  arXiv  Detail & Related papers  (2020-07-20T18:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.