Spiking Neural Networks with Random Network Architecture

• URL: http://arxiv.org/abs/2505.13622v1
• Date: Mon, 19 May 2025 18:03:55 GMT
• Title: Spiking Neural Networks with Random Network Architecture
• Authors: Zihan Dai, Huanfei Ma,
• Abstract summary: This work proposes a new architecture for spiking neural networks, RanSNN, where only part of the network weights need training and all the classic training methods can be adopted.<n>Compared with traditional training methods for spiking neural networks, it greatly improves the training efficiency while ensuring the training performance, and also has good versatility and stability as validated by benchmark tests.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The spiking neural network, known as the third generation neural network, is an important network paradigm. Due to its mode of information propagation that follows biological rationality, the spiking neural network has strong energy efficiency and has advantages in complex high-energy application scenarios. However, unlike the artificial neural network (ANN) which has a mature and unified framework, the SNN models and training methods have not yet been widely unified due to the discontinuous and non-differentiable property of the firing mechanism. Although several algorithms for training spiking neural networks have been proposed in the subsequent development process, some fundamental issues remain unsolved. Inspired by random network design, this work proposes a new architecture for spiking neural networks, RanSNN, where only part of the network weights need training and all the classic training methods can be adopted. Compared with traditional training methods for spiking neural networks, it greatly improves the training efficiency while ensuring the training performance, and also has good versatility and stability as validated by benchmark tests.

Related papers

• Peer-to-Peer Learning Dynamics of Wide Neural Networks [10.179711440042123]
  We provide an explicit characterization of the learning dynamics of wide neural networks trained using distributed gradient descent (DGD) algorithms.<n>Our results leverage both recent in neural tangent kernel (NTK) theory and extensive previous work on distributed learning and consensus tasks.
  arXiv  Detail & Related papers  (2024-09-23T17:57:58Z)
• Message Passing Variational Autoregressive Network for Solving Intractable Ising Models [6.261096199903392]
  Many deep neural networks have been used to solve Ising models, including autoregressive neural networks, convolutional neural networks, recurrent neural networks, and graph neural networks. Here we propose a variational autoregressive architecture with a message passing mechanism, which can effectively utilize the interactions between spin variables. The new network trained under an annealing framework outperforms existing methods in solving several prototypical Ising spin Hamiltonians, especially for larger spin systems at low temperatures.
  arXiv  Detail & Related papers  (2024-04-09T11:27:07Z)
• NeuralFastLAS: Fast Logic-Based Learning from Raw Data [54.938128496934695]
  Symbolic rule learners generate interpretable solutions, however they require the input to be encoded symbolically. Neuro-symbolic approaches overcome this issue by mapping raw data to latent symbolic concepts using a neural network. We introduce NeuralFastLAS, a scalable and fast end-to-end approach that trains a neural network jointly with a symbolic learner.
  arXiv  Detail & Related papers  (2023-10-08T12:33:42Z)
• Spike-based computation using classical recurrent neural networks [1.9171404264679484]
  Spiking neural networks are artificial neural networks in which communication between neurons is only made of events, also called spikes. We modify the dynamics of a well-known, easily trainable type of recurrent neural network to make it event-based. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark.
  arXiv  Detail & Related papers  (2023-06-06T12:19:12Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• A Hybrid Neural Coding Approach for Pattern Recognition with Spiking Neural Networks [53.31941519245432]
  Brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. These SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes.
  arXiv  Detail & Related papers  (2023-05-26T02:52:12Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• An SMT-Based Approach for Verifying Binarized Neural Networks [1.4394939014120451]
  We propose an SMT-based technique for verifying Binarized Neural Networks. One novelty of our technique is that it allows the verification of neural networks that include both binarized and non-binarized components. We implement our technique as an extension to the Marabou framework, and use it to evaluate the approach on popular binarized neural network architectures.
  arXiv  Detail & Related papers  (2020-11-05T16:21:26Z)
• Stochastic Markov Gradient Descent and Training Low-Bit Neural Networks [77.34726150561087]
  We introduce Gradient Markov Descent (SMGD), a discrete optimization method applicable to training quantized neural networks. We provide theoretical guarantees of algorithm performance as well as encouraging numerical results.
  arXiv  Detail & Related papers  (2020-08-25T15:48:15Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks [0.9790524827475205]
  We show how a novel type of adaptive spiking recurrent neural network (SRNN) is able to achieve state-of-the-art performance. We calculate a $>$100x energy improvement for our SRNNs over classical RNNs on the harder tasks.
  arXiv  Detail & Related papers  (2020-05-24T01:04:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.