Finite Meta-Dynamic Neurons in Spiking Neural Networks for Spatio-temporal Learning

• URL: http://arxiv.org/abs/2010.03140v1
• Date: Wed, 7 Oct 2020 03:49:28 GMT
• Title: Finite Meta-Dynamic Neurons in Spiking Neural Networks for Spatio-temporal Learning
• Authors: Xiang Cheng and Tielin Zhang and Shuncheng Jia and Bo Xu
• Abstract summary: Spiking Neural Networks (SNNs) have incorporated more biologically-plausible structures and learning principles. We propose Meta-Dynamic Neurons (MDNs) to improve SNNs for a better network generalization during-temporal learning. The MDNs generated from a spatial (MNIST) and a temporal (TIts) datasets first and then extended to various other different-temporal tasks.
• Score: 13.037452551907657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking Neural Networks (SNNs) have incorporated more biologically-plausible structures and learning principles, hence are playing critical roles in bridging the gap between artificial and natural neural networks. The spikes are the sparse signals describing the above-threshold event-based firing and under-threshold dynamic computation of membrane potentials, which give us an alternative uniformed and efficient way on both information representation and computation. Inspired from the biological network, where a finite number of meta neurons integrated together for various of cognitive functions, we proposed and constructed Meta-Dynamic Neurons (MDN) to improve SNNs for a better network generalization during spatio-temporal learning. The MDNs are designed with basic neuronal dynamics containing 1st-order and 2nd-order dynamics of membrane potentials, including the spatial and temporal meta types supported by some hyper-parameters. The MDNs generated from a spatial (MNIST) and a temporal (TIDigits) datasets first, and then extended to various other different spatio-temporal tasks (including Fashion-MNIST, NETtalk, Cifar-10, TIMIT and N-MNIST). The comparable accuracy was reached compared to other SOTA SNN algorithms, and a better generalization was also achieved by SNNs using MDNs than that without using MDNs.

Related papers

• Spatial-Temporal Search for Spiking Neural Networks [32.937536365872745]
  Spiking Neural Networks (SNNs) are considered as a potential candidate for the next generation of artificial intelligence. We propose a differentiable approach to optimize SNN on both spatial and temporal dimensions. Our methods achieve comparable classification performance of CIFAR10/100 and ImageNet with accuracies of 96.43%, 78.96%, and 70.21%, respectively.
  arXiv  Detail & Related papers  (2024-10-24T09:32:51Z)
• Scalable Mechanistic Neural Networks [52.28945097811129]
  We propose an enhanced neural network framework designed for scientific machine learning applications involving long temporal sequences. By reformulating the original Mechanistic Neural Network (MNN) we reduce the computational time and space complexities from cubic and quadratic with respect to the sequence length, respectively, to linear. Extensive experiments demonstrate that S-MNN matches the original MNN in precision while substantially reducing computational resources.
  arXiv  Detail & Related papers  (2024-10-08T14:27:28Z)
• Direct Training High-Performance Deep Spiking Neural Networks: A Review of Theories and Methods [33.377770671553336]
  Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks (ANNs) In this paper, we provide a new perspective to summarize the theories and methods for training deep SNNs with high performance.
  arXiv  Detail & Related papers  (2024-05-06T09:58:54Z)
• Stochastic Spiking Neural Networks with First-to-Spike Coding [7.955633422160267]
  Spiking Neural Networks (SNNs) are known for their bio-plausibility and energy efficiency. In this work, we explore the merger of novel computing and information encoding schemes in SNN architectures. We investigate the tradeoffs of our proposal in terms of accuracy, inference latency, spiking sparsity, energy consumption, and datasets.
  arXiv  Detail & Related papers  (2024-04-26T22:52:23Z)
• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Multi-scale Evolutionary Neural Architecture Search for Deep Spiking Neural Networks [7.271032282434803]
  We propose a Multi-Scale Evolutionary Neural Architecture Search (MSE-NAS) for Spiking Neural Networks (SNNs) MSE-NAS evolves individual neuron operation, self-organized integration of multiple circuit motifs, and global connectivity across motifs through a brain-inspired indirect evaluation function, Representational Dissimilarity Matrices (RDMs) The proposed algorithm achieves state-of-the-art (SOTA) performance with shorter simulation steps on static datasets and neuromorphic datasets.
  arXiv  Detail & Related papers  (2023-04-21T05:36:37Z)
• Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Study [55.12108376616355]
  The study on NTK has been devoted to typical neural network architectures, but is incomplete for neural networks with Hadamard products (NNs-Hp) In this work, we derive the finite-width-K formulation for a special class of NNs-Hp, i.e., neural networks. We prove their equivalence to the kernel regression predictor with the associated NTK, which expands the application scope of NTK.
  arXiv  Detail & Related papers  (2022-09-16T06:36:06Z)
• A Synapse-Threshold Synergistic Learning Approach for Spiking Neural Networks [1.8556712517882232]
  Spiking neural networks (SNNs) have demonstrated excellent capabilities in various intelligent scenarios. In this study, we develop a novel synergistic learning approach that involves simultaneously training synaptic weights and spike thresholds in SNNs.
  arXiv  Detail & Related papers  (2022-06-10T06:41:36Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.