Related papers: A Digital Machine Learning Algorithm Simulating Spiking Neural Network CoLaNET

A Digital Machine Learning Algorithm Simulating Spiking Neural Network CoLaNET

URL: http://arxiv.org/abs/2503.17111v1
Date: Fri, 21 Mar 2025 12:55:24 GMT
Title: A Digital Machine Learning Algorithm Simulating Spiking Neural Network CoLaNET
Authors: Mikhail Kiselev,
Abstract summary: CoLaNET (columnar layered network) SNN architecture was invented.<n>The distinctive feature of this architecture is a combination of prototypical network structures corresponding to different classes.<n>To make it easier, I developed a continuous numeric (non-spiking) machine learning algorithm which approximates CoLaNET behavior with satisfactory accuracy.
Score: 0.0
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
Abstract: During last several years, our research team worked on development of a spiking neural network (SNN) architecture, which could be used in the wide range of supervised learning classification tasks. It should work under the condition, that all participating signals (the classified object description, correct class label and SNN decision) should have spiking nature. As a result, the CoLaNET (columnar layered network) SNN architecture was invented. The distinctive feature of this architecture is a combination of prototypical network structures corresponding to different classes and significantly distinctive instances of one class (=columns) and functionally differing populations of neurons inside columns (=layers). The other distinctive feature is a novel combination of anti-Hebbian and dopamine-modulated plasticity. While CoLaNET is relatively simple, it includes several hyperparameters. Their choice for particular classification tasks is not trivial. Besides that, specific features of the data classified (e.g. classification of separate pictures like in MNIST dataset vs. classifying objects in a continuous video stream) require certain modifications of CoLaNET structure. To solve these problems, the deep mathematical exploration of CoLaNET should be carried out. However, SNNs, being stochastic discrete systems, are usually very hard for exact mathematical analysis. To make it easier, I developed a continuous numeric (non-spiking) machine learning algorithm which approximates CoLaNET behavior with satisfactory accuracy. It is described in the paper. At present, it is being studied by exact analytic methods. We hope that the results of this study could be applied to direct calculation of CoLaNET hyperparameters and optimization of its structure.

Related papers

CoLaNET -- A Spiking Neural Network with Columnar Layered Architecture for Classification [0.0]
I describe a spiking neural network (SNN) architecture which, can be used in wide range of supervised learning classification tasks.<n>It is assumed, that all participating signals (the classified object description, correct class label and SNN decision) have spiking nature.<n>I illustrate the high performance of my network on a task related to model-based reinforcement learning.
arXiv Detail & Related papers (2024-09-02T13:04:54Z)
Unveiling the Power of Sparse Neural Networks for Feature Selection [60.50319755984697]
Sparse Neural Networks (SNNs) have emerged as powerful tools for efficient feature selection. We show that SNNs trained with dynamic sparse training (DST) algorithms can achieve, on average, more than $50%$ memory and $55%$ FLOPs reduction. Our findings show that feature selection with SNNs trained with DST algorithms can achieve, on average, more than $50%$ memory and $55%$ FLOPs reduction.
arXiv Detail & Related papers (2024-08-08T16:48:33Z)
Time Elastic Neural Networks [2.1756081703276]
We introduce and detail an atypical neural network architecture, called time elastic neural network (teNN) The novelty compared to classical neural network architecture is that it explicitly incorporates time warping ability. We demonstrate that, during the training process, the teNN succeeds in reducing the number of neurons required within each cell.
arXiv Detail & Related papers (2024-05-27T09:01:30Z)
How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model [47.617093812158366]
We introduce the Random Hierarchy Model: a family of synthetic tasks inspired by the hierarchical structure of language and images. We find that deep networks learn the task by developing internal representations invariant to exchanging equivalent groups. Our results indicate how deep networks overcome the curse of dimensionality by building invariant representations.
arXiv Detail & Related papers (2023-07-05T09:11:09Z)
Neural Attentive Circuits [93.95502541529115]
We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs achieve an 8x speedup at inference time while losing less than 3% performance.
arXiv Detail & Related papers (2022-10-14T18:00:07Z)
Dynamical systems' based neural networks [0.7874708385247353]
We build neural networks using a suitable, structure-preserving, numerical time-discretisation. The structure of the neural network is then inferred from the properties of the ODE vector field. We present two universal approximation results and demonstrate how to impose some particular properties on the neural networks.
arXiv Detail & Related papers (2022-10-05T16:30:35Z)
Towards a General Purpose CNN for Long Range Dependencies in $\mathrm{N}$D [49.57261544331683]
We propose a single CNN architecture equipped with continuous convolutional kernels for tasks on arbitrary resolution, dimensionality and length without structural changes. We show the generality of our approach by applying the same CCNN to a wide set of tasks on sequential (1$mathrmD$) and visual data (2$mathrmD$) Our CCNN performs competitively and often outperforms the current state-of-the-art across all tasks considered.
arXiv Detail & Related papers (2022-06-07T15:48:02Z)
Neural networks with linear threshold activations: structure and algorithms [1.795561427808824]
We show that 2 hidden layers are necessary and sufficient to represent any function representable in the class. We also give precise bounds on the sizes of the neural networks required to represent any function in the class. We propose a new class of neural networks that we call shortcut linear threshold networks.
arXiv Detail & Related papers (2021-11-15T22:33:52Z)
Neural network relief: a pruning algorithm based on neural activity [47.57448823030151]
We propose a simple importance-score metric that deactivates unimportant connections. We achieve comparable performance for LeNet architectures on MNIST. The algorithm is not designed to minimize FLOPs when considering current hardware and software implementations.
arXiv Detail & Related papers (2021-09-22T15:33:49Z)
The Separation Capacity of Random Neural Networks [78.25060223808936]
We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability. We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
arXiv Detail & Related papers (2021-07-31T10:25:26Z)
Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity [87.32169414230822]
Recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) In this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs.
arXiv Detail & Related papers (2020-08-21T19:03:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.