A Faster Approach to Spiking Deep Convolutional Neural Networks

• URL: http://arxiv.org/abs/2210.17442v1
• Date: Mon, 31 Oct 2022 16:13:15 GMT
• Title: A Faster Approach to Spiking Deep Convolutional Neural Networks
• Authors: Shahriar Rezghi Shirsavar (University of Tehran, Iran), Mohammad-Reza A. Dehaqani (University of Tehran, Iran)
• Abstract summary: Spiking neural networks (SNNs) have closer dynamics to the brain than current deep neural networks. We propose a network structure based on previous work to improve network runtime and accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking neural networks (SNNs) have closer dynamics to the brain than current deep neural networks. Their low power consumption and sample efficiency make these networks interesting. Recently, several deep convolutional spiking neural networks have been proposed. These networks aim to increase biological plausibility while creating powerful tools to be applied to machine learning tasks. Here, we suggest a network structure based on previous work to improve network runtime and accuracy. Improvements to the network include reducing training iterations to only once, effectively using principal component analysis (PCA) dimension reduction, weight quantization, timed outputs for classification, and better hyperparameter tuning. Furthermore, the preprocessing step is changed to allow the processing of colored images instead of only black and white to improve accuracy. The proposed structure fractionalizes runtime and introduces an efficient approach to deep convolutional SNNs.

Related papers

• Towards Efficient Deep Spiking Neural Networks Construction with Spiking Activity based Pruning [17.454100169491497]
  We propose a structured pruning approach based on the activity levels of convolutional kernels named Spiking Channel Activity-based (SCA) network pruning framework. Inspired by synaptic plasticity mechanisms, our method dynamically adjusts the network's structure by pruning and regenerating convolutional kernels during training, enhancing the model's adaptation to the current target task.
  arXiv  Detail & Related papers  (2024-06-03T07:44:37Z)
• Efficient and Flexible Method for Reducing Moderate-size Deep Neural Networks with Condensation [36.41451383422967]
  In scientific applications, the scale of neural networks is generally moderate-size, mainly to ensure the speed of inference. Existing work has found that the powerful capabilities of neural networks are primarily due to their non-linearity. We propose a condensation reduction algorithm to verify the feasibility of this idea in practical problems.
  arXiv  Detail & Related papers  (2024-05-02T06:53:40Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Intelligence Processing Units Accelerate Neuromorphic Learning [52.952192990802345]
  Spiking neural networks (SNNs) have achieved orders of magnitude improvement in terms of energy consumption and latency. We present an IPU-optimized release of our custom SNN Python package, snnTorch.
  arXiv  Detail & Related papers  (2022-11-19T15:44:08Z)
• Optimizing the Consumption of Spiking Neural Networks with Activity Regularization [15.317534913990633]
  Spiking Neural Networks (SNNs) are an example of bio-inspired techniques that can further save energy by using binary activations, and avoid consuming energy when not spiking. In this work, we look into different techniques to enforce sparsity on the neural network activation maps and compare the effect of different training regularizers on the efficiency of the optimized DNNs and SNNs.
  arXiv  Detail & Related papers  (2022-04-04T13:19:47Z)
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision [64.71260357476602]
  Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks. We propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection.
  arXiv  Detail & Related papers  (2021-12-06T23:45:58Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• A Light-Weighted Convolutional Neural Network for Bitemporal SAR Image Change Detection [40.58864817923371]
  We propose a lightweight neural network to reduce the computational and spatial complexity. In the proposed network, we replace normal convolutional layers with bottleneck layers that keep the same number of channels between input and output. We verify our light-weighted neural network on four sets of bitemporal SAR images.
  arXiv  Detail & Related papers  (2020-05-29T04:01:32Z)
• Network Adjustment: Channel Search Guided by FLOPs Utilization Ratio [101.84651388520584]
  This paper presents a new framework named network adjustment, which considers network accuracy as a function of FLOPs. Experiments on standard image classification datasets and a wide range of base networks demonstrate the effectiveness of our approach.
  arXiv  Detail & Related papers  (2020-04-06T15:51:00Z)
• Exploring the Connection Between Binary and Spiking Neural Networks [1.329054857829016]
  We bridge the recent algorithmic progress in training Binary Neural Networks and Spiking Neural Networks. We show that training Spiking Neural Networks in the extreme quantization regime results in near full precision accuracies on large-scale datasets.
  arXiv  Detail & Related papers  (2020-02-24T03:46:51Z)
• Large-Scale Gradient-Free Deep Learning with Recursive Local Representation Alignment [84.57874289554839]
  Training deep neural networks on large-scale datasets requires significant hardware resources. Backpropagation, the workhorse for training these networks, is an inherently sequential process that is difficult to parallelize. We propose a neuro-biologically-plausible alternative to backprop that can be used to train deep networks.
  arXiv  Detail & Related papers  (2020-02-10T16:20:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.