A Faster Approach to Spiking Deep Convolutional Neural Networks

• URL: http://arxiv.org/abs/2210.17442v1
• Date: Mon, 31 Oct 2022 16:13:15 GMT
• Title: A Faster Approach to Spiking Deep Convolutional Neural Networks
• Authors: Shahriar Rezghi Shirsavar (University of Tehran, Iran), Mohammad-Reza A. Dehaqani (University of Tehran, Iran)
• Abstract summary: Spiking neural networks (SNNs) have closer dynamics to the brain than current deep neural networks. We propose a network structure based on previous work to improve network runtime and accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking neural networks (SNNs) have closer dynamics to the brain than current deep neural networks. Their low power consumption and sample efficiency make these networks interesting. Recently, several deep convolutional spiking neural networks have been proposed. These networks aim to increase biological plausibility while creating powerful tools to be applied to machine learning tasks. Here, we suggest a network structure based on previous work to improve network runtime and accuracy. Improvements to the network include reducing training iterations to only once, effectively using principal component analysis (PCA) dimension reduction, weight quantization, timed outputs for classification, and better hyperparameter tuning. Furthermore, the preprocessing step is changed to allow the processing of colored images instead of only black and white to improve accuracy. The proposed structure fractionalizes runtime and introduces an efficient approach to deep convolutional SNNs.

Related papers

• Exploring Neural Network Pruning with Screening Methods [3.443622476405787]
  Modern deep learning models have tens of millions of parameters which makes the inference processes resource-intensive. This paper proposes and evaluates a network pruning framework that eliminates non-essential parameters. The proposed framework produces competitive lean networks compared to the original networks.
  arXiv  Detail & Related papers  (2025-02-11T02:31:04Z)
• When less is more: evolving large neural networks from small ones [0.0]
  We study feed-forward neural networks that are small and dynamic, whose nodes can be added (or subtracted) during training. A single neuronal weight in the network controls the network's size, while the weight itself is optimized by the same gradient-descent algorithm. We train and evaluate such Nimble Neural Networks on nonlinear regression and classification tasks where they outperform the corresponding static networks.
  arXiv  Detail & Related papers  (2025-01-29T21:56:38Z)
• Deep-Unrolling Multidimensional Harmonic Retrieval Algorithms on Neuromorphic Hardware [78.17783007774295]
  This paper explores the potential of conversion-based neuromorphic algorithms for highly accurate and energy-efficient single-snapshot multidimensional harmonic retrieval. A novel method for converting the complex-valued convolutional layers and activations into spiking neural networks (SNNs) is developed. The converted SNNs achieve almost five-fold power efficiency at moderate performance loss compared to the original CNNs.
  arXiv  Detail & Related papers  (2024-12-05T09:41:33Z)
• Efficient and Flexible Method for Reducing Moderate-size Deep Neural Networks with Condensation [36.41451383422967]
  In scientific applications, the scale of neural networks is generally moderate-size, mainly to ensure the speed of inference. Existing work has found that the powerful capabilities of neural networks are primarily due to their non-linearity. We propose a condensation reduction algorithm to verify the feasibility of this idea in practical problems.
  arXiv  Detail & Related papers  (2024-05-02T06:53:40Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• A Light-Weighted Convolutional Neural Network for Bitemporal SAR Image Change Detection [40.58864817923371]
  We propose a lightweight neural network to reduce the computational and spatial complexity. In the proposed network, we replace normal convolutional layers with bottleneck layers that keep the same number of channels between input and output. We verify our light-weighted neural network on four sets of bitemporal SAR images.
  arXiv  Detail & Related papers  (2020-05-29T04:01:32Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)
• Exploring the Connection Between Binary and Spiking Neural Networks [1.329054857829016]
  We bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets.
  arXiv  Detail & Related papers  (2020-02-24T03:46:51Z)
• Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment [84.57874289554839]
  Training deep neural networks on large-scale datasets requires significant hardware resources. Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize. We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
  arXiv  Detail & Related papers  (2020-02-10T16:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.