Explosive neural networks via higher-order interactions in curved statistical manifolds

• URL: http://arxiv.org/abs/2408.02326v2
• Date: Tue, 04 Feb 2025 09:13:46 GMT
• Title: Explosive neural networks via higher-order interactions in curved statistical manifolds
• Authors: Miguel Aguilera, Pablo A. Morales, Fernando E. Rosas, Hideaki Shimazaki,
• Abstract summary: We introduce curved neural networks as a class of prototypical models with a limited number of parameters. We show that these curved neural networks implement a self-regulating process that can accelerate memory retrieval. We analytically explore their memory-retrieval capacity using the replica trick near ferromagnetic and spin-glass phase boundaries.
• Score: 43.496401697112695
• License:
• Abstract: Higher-order interactions underlie complex phenomena in systems such as biological and artificial neural networks, but their study is challenging due to the scarcity of tractable models. By leveraging a generalisation of the maximum entropy principle, here we introduce curved neural networks as a class of prototypical models with a limited number of parameters that are particularly well-suited for studying higher-order phenomena. Through exact mean-field descriptions, we show that these curved neural networks implement a self-regulating annealing process that can accelerate memory retrieval, leading to explosive order-disorder phase transitions with multi-stability and hysteresis effects. Moreover, by analytically exploring their memory-retrieval capacity using the replica trick near ferromagnetic and spin-glass phase boundaries, we demonstrate that these networks can enhance memory capacity and robustness of retrieval over classical associative-memory networks. Overall, the proposed framework provides parsimonious models amenable to analytical study, revealing novel higher-order phenomena in complex networks.

Related papers

• Storing overlapping associative memories on latent manifolds in low-rank spiking networks [5.041384008847852]
  We revisit the associative memory problem in light of advances in understanding spike-based computation. We show that the spiking activity for a large class of all-inhibitory networks is situated on a low-dimensional, convex, and piecewise-linear manifold. We propose several learning rules, and demonstrate a linear scaling of the storage capacity with the number of neurons, as well as robust pattern completion abilities.
  arXiv  Detail & Related papers  (2024-11-26T14:48:25Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Long Sequence Hopfield Memory [32.28395813801847]
  Sequence memory enables agents to encode, store, and retrieve complex sequences of stimuli and actions. We introduce a nonlinear interaction term, enhancing separation between the patterns. We extend this model to store sequences with variable timing between states' transitions.
  arXiv  Detail & Related papers  (2023-06-07T15:41:03Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Physically constrained neural networks to solve the inverse problem for neuron models [0.29005223064604074]
  Systems biology and systems neurophysiology are powerful tools for a number of key applications in the biomedical sciences. Recent developments in the field of deep neural networks have demonstrated the possibility of formulating nonlinear, universal approximators.
  arXiv  Detail & Related papers  (2022-09-24T12:51:15Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)
• A new inference approach for training shallow and deep generalized linear models of noisy interacting neurons [4.899818550820575]
  We develop a two-step inference strategy that allows us to train robust generalized linear models of interacting neurons. We show that, compared to classical methods, the models trained in this way exhibit improved performance. The method can be extended to deep convolutional neural networks, leading to models with high predictive accuracy for both the neuron firing rates and their correlations.
  arXiv  Detail & Related papers  (2020-06-11T15:09:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.