Explosive neural networks via higher-order interactions in curved statistical manifolds

• URL: http://arxiv.org/abs/2408.02326v1
• Date: Mon, 5 Aug 2024 09:10:29 GMT
• Title: Explosive neural networks via higher-order interactions in curved statistical manifolds
• Authors: Miguel Aguilera, Pablo A. Morales, Fernando E. Rosas, Hideaki Shimazaki,
• Abstract summary: We introduce curved neural networks as a class of prototypical models for studying higher-order phenomena. We show that these curved neural networks implement a self-regulating process that can accelerate memory retrieval.
• Score: 43.496401697112695
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Higher-order interactions underlie complex phenomena in systems such as biological and artificial neural networks, but their study is challenging due to the lack of tractable standard models. By leveraging the maximum entropy principle in curved statistical manifolds, here we introduce curved neural networks as a class of prototypical models for studying higher-order phenomena. Through exact mean-field descriptions, we show that these curved neural networks implement a self-regulating annealing process that can accelerate memory retrieval, leading to explosive order-disorder phase transitions with multi-stability and hysteresis effects. Moreover, by analytically exploring their memory capacity using the replica trick near ferromagnetic and spin-glass phase boundaries, we demonstrate that these networks enhance memory capacity over the classical associative-memory networks. Overall, the proposed framework provides parsimonious models amenable to analytical study, revealing novel higher-order phenomena in complex network systems.

Related papers

• Storing overlapping associative memories on latent manifolds in low-rank spiking networks [5.041384008847852]
  We revisit the associative memory problem in light of advances in understanding spike-based computation. We show that the spiking activity for a large class of all-inhibitory networks is situated on a low-dimensional, convex, and piecewise-linear manifold. We propose several learning rules, and demonstrate a linear scaling of the storage capacity with the number of neurons, as well as robust pattern completion abilities.
  arXiv  Detail & Related papers  (2024-11-26T14:48:25Z)
• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Attention for Causal Relationship Discovery from Biological Neural Dynamics [9.097847269529202]
  This paper explores the potential of the transformer models for learning Granger causality in networks with complex nonlinear dynamics at every node. We show that the cross attention module effectively captures the causal relationship among neurons, with an accuracy equal or superior to that for the most popular Granger causality analysis method.
  arXiv  Detail & Related papers  (2023-11-12T18:59:42Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Hyperbolic Neural Networks++ [66.16106727715061]
  We generalize the fundamental components of neural networks in a single hyperbolic geometry model, namely, the Poincar'e ball model. Experiments show the superior parameter efficiency of our methods compared to conventional hyperbolic components, and stability and outperformance over their Euclidean counterparts.
  arXiv  Detail & Related papers  (2020-06-15T08:23:20Z)
• A new inference approach for training shallow and deep generalized linear models of noisy interacting neurons [4.899818550820575]
  We develop a two-step inference strategy that allows us to train robust generalized linear models of interacting neurons. We show that, compared to classical methods, the models trained in this way exhibit improved performance. The method can be extended to deep convolutional neural networks, leading to models with high predictive accuracy for both the neuron firing rates and their correlations.
  arXiv  Detail & Related papers  (2020-06-11T15:09:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.