Low-activity supervised convolutional spiking neural networks applied to speech commands recognition

• URL: http://arxiv.org/abs/2011.06846v1
• Date: Fri, 13 Nov 2020 10:29:35 GMT
• Title: Low-activity supervised convolutional spiking neural networks applied to speech commands recognition
• Authors: Thomas Pellegrini, Romain Zimmer, Timoth\'ee Masquelier
• Abstract summary: Spiking Neural Networks (SNNs) can be trained efficiently in a supervised manner. We show that a model comprised of stacked dilated convolution spiking layers can reach an error rate very close to standard Deep Neural Networks (DNNs) We also show that modeling the leakage of the neuron membrane potential is useful, since the LIF model outperformed its non-leaky model counterpart significantly.
• Score: 6.6389732792316005
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Deep Neural Networks (DNNs) are the current state-of-the-art models in many speech related tasks. There is a growing interest, though, for more biologically realistic, hardware friendly and energy efficient models, named Spiking Neural Networks (SNNs). Recently, it has been shown that SNNs can be trained efficiently, in a supervised manner, using backpropagation with a surrogate gradient trick. In this work, we report speech command (SC) recognition experiments using supervised SNNs. We explored the Leaky-Integrate-Fire (LIF) neuron model for this task, and show that a model comprised of stacked dilated convolution spiking layers can reach an error rate very close to standard DNNs on the Google SC v1 dataset: 5.5%, while keeping a very sparse spiking activity, below 5%, thank to a new regularization term. We also show that modeling the leakage of the neuron membrane potential is useful, since the LIF model outperformed its non-leaky model counterpart significantly.

Related papers

• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• Fully Spiking Denoising Diffusion Implicit Models [61.32076130121347]
  Spiking neural networks (SNNs) have garnered considerable attention owing to their ability to run on neuromorphic devices with super-high speeds. We propose a novel approach fully spiking denoising diffusion implicit model (FSDDIM) to construct a diffusion model within SNNs. We demonstrate that the proposed method outperforms the state-of-the-art fully spiking generative model.
  arXiv  Detail & Related papers  (2023-12-04T09:07:09Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean [1.0923877073891446]
  Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks. Despite this success, the inner workings of DNNs are often not transparent. This lack of interpretability has led to increased research on inherently interpretable neural networks.
  arXiv  Detail & Related papers  (2023-01-27T17:06:13Z)
• Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics [6.309365332210523]
  Spiking Neural Networks (SNNs) with their neuro-inspired event-driven processing can efficiently handle asynchronous data. We propose an adaptive fully-spiking framework with learnable neuronal dynamics to alleviate the spike vanishing problem. Our experiments on datasets show an average reduction of 13% in average endpoint error (AEE) compared to state-of-the-art ANNs.
  arXiv  Detail & Related papers  (2022-09-21T21:17:56Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Deep Learning in Spiking Phasor Neural Networks [0.6767885381740952]
  Spiking Neural Networks (SNNs) have attracted the attention of the deep learning community for use in low-latency, low-power neuromorphic hardware. In this paper, we introduce Spiking Phasor Neural Networks (SPNNs) SPNNs are based on complex-valued Deep Neural Networks (DNNs), representing phases by spike times.
  arXiv  Detail & Related papers  (2022-04-01T15:06:15Z)
• Event-based Video Reconstruction via Potential-assisted Spiking Neural Network [48.88510552931186]
  Bio-inspired neural networks can potentially lead to greater computational efficiency on event-driven hardware. We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN) We find that the spiking neurons have the potential to store useful temporal information (memory) to complete such time-dependent tasks.
  arXiv  Detail & Related papers  (2022-01-25T02:05:20Z)
• Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments [24.754429120321365]
  Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies. One promising area for high performance SNNs is template matching and image recognition. This research introduces the first high performance SNN for the Visual Place Recognition (VPR) task.
  arXiv  Detail & Related papers  (2021-09-14T05:40:40Z)
• Spiking Neural Networks with Single-Spike Temporal-Coded Neurons for Network Intrusion Detection [6.980076213134383]
  Spiking neural network (SNN) is interesting due to its strong bio-plausibility and high energy efficiency. However, its performance is falling far behind conventional deep neural networks (DNNs)
  arXiv  Detail & Related papers  (2020-10-15T14:46:18Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.