Geometric and dynamical analysis of attractor boundaries and storage limits in kernel Hopfield networks

• URL: http://arxiv.org/abs/2605.00366v2
• Date: Fri, 08 May 2026 13:14:56 GMT
• Title: Geometric and dynamical analysis of attractor boundaries and storage limits in kernel Hopfield networks
• Abstract summary: This paper investigates the global geometry of attractor basins and the mechanisms governing the storage limit in KLR-trained Hopfield networks.<n>Experiments show that the network achieves a storage capacity for random sequences up to $P/N approx 16$, while maintaining stable retrieval for structured data at effective loads near $P/N approx 20$.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-capacity associative memories based on Kernel Logistic Regression (KLR) exhibit strong storage capabilities, but the dynamical and geometric mechanisms underlying their stability remain poorly understood. This paper investigates the global geometry of attractor basins and the mechanisms governing the storage limit in KLR-trained Hopfield networks. We combine empirical evaluations using random sequences and real-world image embeddings (CIFAR-10) with morphing experiments and statistical Signal-to-Noise Ratio (SNR) analysis. Our experiments show that the network achieves a storage capacity for random sequences up to $P/N \approx 16$, while maintaining stable retrieval for structured data at effective loads near $P/N \approx 20$. Morphing analysis indicates that attractors on the "Ridge of Optimization" are separated by sharp, phase-transition-like boundaries, characterized by steep effective potential barriers and critical slowing down. Furthermore, by comparing an SNR analysis with a geometric reference point inspired by Cover's theorem, we show that the practical storage limit is governed primarily not by a lack of geometric separability in the feature space, but by the loss of dynamical stability against crosstalk noise. These findings suggest that KLR networks function as highly localized exemplar-based memories that operate near the onset of dynamical collapse, providing a useful perspective on the design of robust, large-scale retrieval systems.

Related papers

• Information Geometric Self-Organization at the Edge of Stability in High-Capacity Kernel Associative Memories [0.0]
  We show that optimal high-capacity memory representations are not formed in flat minima, but are sculpted at the highly curved boundaries of geometric singularities.<n>These findings suggest that optimal, high-capacity memory representations are not formed in flat minima, but are dynamically sculpted at the highly curved boundaries of geometric singularities.
  arXiv  Detail & Related papers  (2026-09-15T08:27:55Z)
• Robust Auto-associative Memory via Convolutional Restricted Hopfield Networks [0.22940141855172036]
  Associative memory models play a fundamental role in pattern retrieval.<n>Existing approaches, including Modern Hopfield Networks (MHNs), and Predictive Coding Networks (PCNs) exhibit limitations in balancing storage capacity, computational efficiency, and robustness.<n>We propose a Convolutional Restricted Hopfield Networks (CRHNs), which integrates convolutional feature extraction with attractor-based memory retrieval in a structured latent space.
  arXiv  Detail & Related papers  (2026-06-11T21:25:54Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• TIDE: Asymmetric Neural Circuits for Stabilized Temporal Inhibitory-Excitatory Dynamics [51.37317469455798]
  Recent Continuous Thought Machine architecture decouples internal computation from external inputs via neural dynamics.<n>We propose to model neural dynamics using asymmetric Excitatory-Inhibitory (E-I) networks.<n>We introduce Temporal Inhibitory-Excitatory Dynamic Engine (TIDE), a neuro-inspired architecture that computes internal representations through neural dynamics stabilized.
  arXiv  Detail & Related papers  (2026-05-19T05:59:13Z)
• Efficient event-driven retrieval in high-capacity kernel Hopfield networks [0.0]
  High-capacity associative memory models, such as Kernel Logistic Regression (KLR) Hopfield networks, have demonstrated strong storage capabilities but typically rely on computationally expensive synchronous updates.<n>We show that, under appropriately tuned kernel parameters, sequential updates exhibit trajectories that are statistically indistinguishable from those of synchronous dynamics.<n>We find that the asynchronous network achieves empirical storage capacities approaching $P/N approx 30$ in static random pattern regimes, exceeding classical limits.
  arXiv  Detail & Related papers  (2026-05-07T10:21:18Z)
• Alternating Gradient Flow Utility: A Unified Metric for Structural Pruning and Dynamic Routing in Deep Networks [52.153950303594684]
  We propose a decoupled kinetic paradigm inspired by Alternating Gradient Flow (AGF)<n>AGF accurately captures the network's structural "kinetic utility"<n>We design a hybrid routing framework that decouples AGF-guided offline structural search from online execution via zero-cost physical priors.
  arXiv  Detail & Related papers  (2026-03-12T18:19:21Z)
• Self-Organization and Spectral Mechanism of Attractor Landscapes in High-Capacity Kernel Hopfield Networks [0.0]
  Kernel-based learning can dramatically increase the storage capacity of Hopfield networks.<n>We show that optimal performance is achieved by tuning the system to a spectral "Goldilocks zone" between rank collapse and diffusion.
  arXiv  Detail & Related papers  (2025-11-17T06:58:34Z)
• Contraction, Criticality, and Capacity: A Dynamical-Systems Perspective on Echo-State Networks [13.857230672081489]
  We present a unified, dynamical-systems treatment that weaves together functional analysis, random attractor theory and recent neuroscientific findings.<n>First, we prove that the Echo-State Property (wash-out of initial conditions) together with global Lipschitz dynamics necessarily yields the Fading-Memory Property.<n>Second, employing a Stone-Weierstrass strategy we give a streamlined proof that ESNs with nonlinear reservoirs and linear read-outs are dense in the Banach space of causal, time-in fading-memory filters.<n>Third, we quantify computational resources via memory-capacity spectrum, show how
  arXiv  Detail & Related papers  (2025-07-24T14:41:18Z)
• Dynamical stability for dense patterns in discrete attractor neural networks [6.159133786557903]
  We derive a theory of the local stability of discrete fixed points in a broad class of networks with graded neural activities and in the presence of noise.<n>Our analysis highlights the computational benefits of threshold-linear activation and sparse-like patterns.
  arXiv  Detail & Related papers  (2025-07-14T15:23:24Z)
• Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization [66.03821840425539]
  In this paper, we investigate the training dynamics of $L$-layer neural networks using the tensor gradient program (SGD) framework.<n>We show that SGD enables these networks to learn linearly independent features that substantially deviate from their initial values.<n>This rich feature space captures relevant data information and ensures that any convergent point of the training process is a global minimum.
  arXiv  Detail & Related papers  (2025-03-12T17:33:13Z)
• Edge of stability echo state networks [5.888495030452654]
  Echo State Networks (ESNs) are time-series processing models working under the Echo State Property (ESP) principle. We introduce a new ESN architecture, called the Edge of Stability Echo State Network (ES$2$N)
  arXiv  Detail & Related papers  (2023-08-05T15:49:25Z)
• Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks [59.142826407441106]
  We study the generalization behavior of shallow neural networks (SNNs) by leveraging the concept of algorithmic stability. We consider gradient descent (GD) and gradient descent (SGD) to train SNNs, for both of which we develop consistent excess bounds.
  arXiv  Detail & Related papers  (2022-09-19T18:48:00Z)
• Convex Analysis of the Mean Field Langevin Dynamics [49.66486092259375]
  convergence rate analysis of the mean field Langevin dynamics is presented. $p_q$ associated with the dynamics allows us to develop a convergence theory parallel to classical results in convex optimization.
  arXiv  Detail & Related papers  (2022-01-25T17:13:56Z)
• Stability of Neural Networks on Manifolds to Relative Perturbations [118.84154142918214]
  Graph Neural Networks (GNNs) show impressive performance in many practical scenarios. GNNs can scale well on large size graphs, but this is contradicted by the fact that existing stability bounds grow with the number of nodes.
  arXiv  Detail & Related papers  (2021-10-10T04:37:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.