Input-driven circuit reconfiguration in critical recurrent neural networks.Marcelo O. Magnasco

• URL: http://arxiv.org/abs/2405.15036v1
• Date: Thu, 23 May 2024 20:15:23 GMT
• Title: Input-driven circuit reconfiguration in critical recurrent neural networks.Marcelo O. Magnasco
• Authors: Marcelo O. Magnasco,
• Abstract summary: We present a very simple single-layer recurrent network, whose pathways can be reconfigured "on fly" using only its inputs. We show this network solves the classical connectedness problem, by allowing signal propagation only along the regions to be evaluated.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Changing a circuit dynamically, without actually changing the hardware itself, is called reconfiguration, and is of great importance due to its manifold technological applications. Circuit reconfiguration appears to be a feature of the cerebral cortex, and hence understanding the neuroarchitectural and dynamical features underlying self-reconfiguration may prove key to elucidate brain function. We present a very simple single-layer recurrent network, whose signal pathways can be reconfigured "on the fly" using only its inputs, with no changes to its synaptic weights. We use the low spatio-temporal frequencies of the input to landscape the ongoing activity, which in turn permits or denies the propagation of traveling waves. This mechanism uses the inherent properties of dynamically-critical systems, which we guarantee through unitary convolution kernels. We show this network solves the classical connectedness problem, by allowing signal propagation only along the regions to be evaluated for connectedness and forbidding it elsewhere.

Related papers

• Leveraging Low-Rank and Sparse Recurrent Connectivity for Robust Closed-Loop Control [63.310780486820796]
  We show how a parameterization of recurrent connectivity influences robustness in closed-loop settings. We find that closed-form continuous-time neural networks (CfCs) with fewer parameters can outperform their full-rank, fully-connected counterparts.
  arXiv  Detail & Related papers  (2023-10-05T21:44:18Z)
• A Step Towards Uncovering The Structure of Multistable Neural Networks [1.14219428942199]
  We study the structure of multistable recurrent neural networks. The activation function is simplified by a nonsmooth Heaviside step function. We derive how multistability is encoded within the network architecture.
  arXiv  Detail & Related papers  (2022-10-06T22:54:17Z)
• Vanilla Feedforward Neural Networks as a Discretization of Dynamical Systems [9.382423715831687]
  In this paper, we back to the classical network structure and prove that the vanilla feedforward networks could also be a numerical discretization of dynamic systems. Our results could provide a new perspective for understanding the approximation properties of feedforward neural networks.
  arXiv  Detail & Related papers  (2022-09-22T10:32:08Z)
• Interference Cancellation GAN Framework for Dynamic Channels [74.22393885274728]
  We introduce an online training framework that can adapt to any changes in the channel. Our framework significantly outperforms recent neural network models on highly dynamic channels.
  arXiv  Detail & Related papers  (2022-08-17T02:01:18Z)
• Cross-Frequency Coupling Increases Memory Capacity in Oscillatory Neural Networks [69.42260428921436]
  Cross-frequency coupling (CFC) is associated with information integration across populations of neurons. We construct a model of CFC which predicts a computational role for observed $theta - gamma$ oscillatory circuits in the hippocampus and cortex. We show that the presence of CFC increases the memory capacity of a population of neurons connected by plastic synapses.
  arXiv  Detail & Related papers  (2022-04-05T17:13:36Z)
• Input correlations impede suppression of chaos and learning in balanced rate networks [58.720142291102135]
  Information encoding and learning in neural circuits depend on how well time-varying stimuli can control spontaneous network activity. We show that in firing-rate networks in the balanced state, external control of recurrent dynamics, strongly depends on correlations in the input.
  arXiv  Detail & Related papers  (2022-01-24T19:20:49Z)
• Relational Self-Attention: What's Missing in Attention for Video Understanding [52.38780998425556]
  We introduce a relational feature transform, dubbed the relational self-attention (RSA) Our experiments and ablation studies show that the RSA network substantially outperforms convolution and self-attention counterparts.
  arXiv  Detail & Related papers  (2021-11-02T15:36:11Z)
• An error-propagation spiking neural network compatible with neuromorphic processors [2.432141667343098]
  We present a spike-based learning method that approximates back-propagation using local weight update mechanisms. We introduce a network architecture that enables synaptic weight update mechanisms to back-propagate error signals. This work represents a first step towards the design of ultra-low power mixed-signal neuromorphic processing systems.
  arXiv  Detail & Related papers  (2021-04-12T07:21:08Z)
• Implicit recurrent networks: A novel approach to stationary input processing with recurrent neural networks in deep learning [0.0]
  In this work, we introduce and test a novel implementation of recurrent neural networks into deep learning. We provide an algorithm which implements the backpropagation algorithm on a implicit implementation of recurrent networks. A single-layer implicit recurrent network is able to solve the XOR problem, while a feed-forward network with monotonically increasing activation function fails at this task.
  arXiv  Detail & Related papers  (2020-10-20T18:55:32Z)
• Theory of gating in recurrent neural networks [5.672132510411465]
  Recurrent neural networks (RNNs) are powerful dynamical models, widely used in machine learning (ML) and neuroscience. Here, we show that gating offers flexible control of two salient features of the collective dynamics. The gate controlling timescales leads to a novel, marginally stable state, where the network functions as a flexible integrator.
  arXiv  Detail & Related papers  (2020-07-29T13:20:58Z)
• Training End-to-End Analog Neural Networks with Equilibrium Propagation [64.0476282000118]
  We introduce a principled method to train end-to-end analog neural networks by gradient descent. We show mathematically that a class of analog neural networks (called nonlinear resistive networks) are energy-based models. Our work can guide the development of a new generation of ultra-fast, compact and low-power neural networks supporting on-chip learning.
  arXiv  Detail & Related papers  (2020-06-02T23:38:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.