Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance

• URL: http://arxiv.org/abs/2108.00706v1
• Date: Mon, 2 Aug 2021 08:20:01 GMT
• Title: Formation of cell assemblies with iterative winners-take-all computation and excitation-inhibition balance
• Authors: Viacheslav Osaulenko and Danylo Ulianych
• Abstract summary: We present an intermediate model that shares the computational ease of kWTA and has more flexible and richer dynamics. We investigate Hebbian-like learning rules and propose a new learning rule for binary weights with multiple stabilization mechanisms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper targets the problem of encoding information into binary cell assemblies. Spiking neural networks and k-winners-take-all models are two common approaches, but the first is hard to use for information processing and the second is too simple and lacks important features of the first. We present an intermediate model that shares the computational ease of kWTA and has more flexible and richer dynamics. It uses explicit inhibitory neurons to balance and shape excitation through an iterative procedure. This leads to a recurrent interaction between inhibitory and excitatory neurons that better adapts to the input distribution and performs such computations as habituation, decorrelation, and clustering. To show these, we investigate Hebbian-like learning rules and propose a new learning rule for binary weights with multiple stabilization mechanisms. Our source code is publicly available.

Related papers

• What to Do When Your Discrete Optimization Is the Size of a Neural Network? [24.546550334179486]
  Machine learning applications using neural networks involve solving discrete optimization problems. classical approaches used in discrete settings do not scale well to large neural networks. We take continuation path (CP) methods to represent using purely the former and Monte Carlo (MC) methods to represent the latter.
  arXiv  Detail & Related papers  (2024-02-15T21:57:43Z)
• Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics [77.34726150561087]
  Recent developments in artificial neural networks, particularly deep learning (DL), are reviewed in detail. Both hybrid and pure machine learning (ML) methods are discussed. History and limitations of AI are recounted and discussed, with particular attention at pointing out misstatements or misconceptions of the classics.
  arXiv  Detail & Related papers  (2022-12-18T02:03:00Z)
• A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithms [64.3064050603721]
  We generalize Runge-Kutta neural network to a recurrent neural network (R2N2) superstructure for the design of customized iterative algorithms. We demonstrate that regular training of the weight parameters inside the proposed superstructure on input/output data of various computational problem classes yields similar iterations to Krylov solvers for linear equation systems, Newton-Krylov solvers for nonlinear equation systems, and Runge-Kutta solvers for ordinary differential equations.
  arXiv  Detail & Related papers  (2022-11-22T16:30:33Z)
• Neural networks trained with SGD learn distributions of increasing complexity [78.30235086565388]
  We show that neural networks trained using gradient descent initially classify their inputs using lower-order input statistics. We then exploit higher-order statistics only later during training. We discuss the relation of DSB to other simplicity biases and consider its implications for the principle of universality in learning.
  arXiv  Detail & Related papers  (2022-11-21T15:27:22Z)
• Understanding Neural Coding on Latent Manifolds by Sharing Features and Dividing Ensembles [3.625425081454343]
  Systems neuroscience relies on two complementary views of neural data, characterized by single neuron tuning curves and analysis of population activity. These two perspectives combine elegantly in neural latent variable models that constrain the relationship between latent variables and neural activity. We propose feature sharing across neural tuning curves, which significantly improves performance and leads to better-behaved optimization.
  arXiv  Detail & Related papers  (2022-10-06T18:37:49Z)
• An Adaptive Contrastive Learning Model for Spike Sorting [12.043679000694258]
  In neuroscience research, it is important to separate out the activity of individual neurons. With the development of large-scale silicon technology, artificially interpreting and labeling spikes is becoming increasingly impractical. We propose a novel modeling framework that learns representations from spikes through contrastive learning.
  arXiv  Detail & Related papers  (2022-05-24T09:18:46Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Units [86.9207811656179]
  We show that memory-augmented neural networks (NNs) can achieve higher-order, memory-augmented extrapolation, stable performance, and faster convergence. Our models achieve a 1.53% average improvement over current state-of-the-art methods in equation verification and achieve a 2.22% Top-1 average accuracy and 2.96% Top-5 average accuracy for equation completion.
  arXiv  Detail & Related papers  (2021-04-07T03:50:11Z)
• BiSNN: Training Spiking Neural Networks with Binary Weights via Bayesian Learning [37.376989855065545]
  Spiking Neural Networks (SNNs) are biologically inspired, dynamic, event-driven models that enhance energy efficiency. An SNN model is introduced that combines the benefits of temporally sparse binary activations and of binary weights. Experiments validate the performance loss with respect to full-precision implementations.
  arXiv  Detail & Related papers  (2020-12-15T14:06:36Z)
• Separation of Memory and Processing in Dual Recurrent Neural Networks [0.0]
  We explore a neural network architecture that stacks a recurrent layer and a feedforward layer that is also connected to the input. When noise is introduced into the activation function of the recurrent units, these neurons are forced into a binary activation regime that makes the networks behave much as finite automata.
  arXiv  Detail & Related papers  (2020-05-17T11:38:42Z)
• Towards Efficient Processing and Learning with Spikes: New Approaches for Multi-Spike Learning [59.249322621035056]
  We propose two new multi-spike learning rules which demonstrate better performance over other baselines on various tasks. In the feature detection task, we re-examine the ability of unsupervised STDP with its limitations being presented. Our proposed learning rules can reliably solve the task over a wide range of conditions without specific constraints being applied.
  arXiv  Detail & Related papers  (2020-05-02T06:41:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.