Improving Spiking Neural Network Accuracy Using Time-based Neurons

• URL: http://arxiv.org/abs/2201.01394v2
• Date: Wed, 2 Mar 2022 07:56:31 GMT
• Title: Improving Spiking Neural Network Accuracy Using Time-based Neurons
• Authors: Hanseok Kim, Woo-Seok Choi
• Abstract summary: Research on neuromorphic computing systems based on low-power spiking neural networks using analog neurons is in the spotlight. As technology scales down, analog neurons are difficult to scale, and they suffer from reduced voltage headroom/dynamic range and circuit nonlinearities. This paper first models the nonlinear behavior of existing current-mirror-based voltage-domain neurons designed in a 28nm process, and show SNN inference accuracy can be severely degraded by the effect of neuron's nonlinearity. We propose a novel neuron, which processes incoming spikes in the time domain and greatly improves the linearity, thereby improving the inference accuracy compared to the
• Score: 0.24366811507669117
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the fundamental limit to reducing power consumption of running deep learning models on von-Neumann architecture, research on neuromorphic computing systems based on low-power spiking neural networks using analog neurons is in the spotlight. In order to integrate a large number of neurons, neurons need to be designed to occupy a small area, but as technology scales down, analog neurons are difficult to scale, and they suffer from reduced voltage headroom/dynamic range and circuit nonlinearities. In light of this, this paper first models the nonlinear behavior of existing current-mirror-based voltage-domain neurons designed in a 28nm process, and show SNN inference accuracy can be severely degraded by the effect of neuron's nonlinearity. Then, to mitigate this problem, we propose a novel neuron, which processes incoming spikes in the time domain and greatly improves the linearity, thereby improving the inference accuracy compared to the existing voltage-domain neuron. Tested on the MNIST dataset, the inference error rate of the proposed neuron differs by less than 0.1% from that of the ideal neuron.

Related papers

• To Spike or Not to Spike, that is the Question [0.45609532372046985]
  SNNs emulate the temporal dynamics of biological neurons, making them well-suited for real-time, event-driven processing. In SNNs, learning rules are based on neurons' spiking behavior, that is, if and when spikes are generated due to a neuron's membrane potential exceeding that neuron's spiking threshold. This work focuses on the significance of neuron learning thresholds alongside weights in SNNs.
  arXiv  Detail & Related papers  (2024-07-28T19:23:09Z)
• The Neuron as a Direct Data-Driven Controller [43.8450722109081]
  This study extends the current normative models, which primarily optimize prediction, by conceptualizing neurons as optimal feedback controllers. We model neurons as biologically feasible controllers which implicitly identify loop dynamics, infer latent states and optimize control. Our model presents a significant departure from the traditional, feedforward, instant-response McCulloch-Pitts-Rosenblatt neuron, offering a novel and biologically-informed fundamental unit for constructing neural networks.
  arXiv  Detail & Related papers  (2024-01-03T01:24:10Z)
• Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity [28.243134476634125]
  In this study, we scrutinized the importance of nonlinear dendrites within neural networks. Our findings reveal that integrating dendritic structures can substantially enhance model capacity and performance.
  arXiv  Detail & Related papers  (2023-06-21T00:28:20Z)
• The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks [64.08042492426992]
  We introduce the Expressive Memory (ELM) neuron model, a biologically inspired model of a cortical neuron. Our ELM neuron can accurately match the aforementioned input-output relationship with under ten thousand trainable parameters. We evaluate it on various tasks with demanding temporal structures, including the Long Range Arena (LRA) datasets.
  arXiv  Detail & Related papers  (2023-06-14T13:34:13Z)
• Constraints on the design of neuromorphic circuits set by the properties of neural population codes [61.15277741147157]
  In the brain, information is encoded, transmitted and used to inform behaviour. Neuromorphic circuits need to encode information in a way compatible to that used by populations of neuron in the brain.
  arXiv  Detail & Related papers  (2022-12-08T15:16:04Z)
• Spiking neural network for nonlinear regression [68.8204255655161]
  Spiking neural networks carry the potential for a massive reduction in memory and energy consumption. They introduce temporal and neuronal sparsity, which can be exploited by next-generation neuromorphic hardware. A framework for regression using spiking neural networks is proposed.
  arXiv  Detail & Related papers  (2022-10-06T13:04:45Z)
• Energy-Efficient High-Accuracy Spiking Neural Network Inference Using Time-Domain Neurons [0.18352113484137625]
  This paper presents a low-power highly linear time-domain I&F neuron circuit. The proposed neuron leads to more than 4.3x lower error rate on the MNIST inference. The power consumed by the proposed neuron circuit is simulated to be 0.230uW per neuron, which is orders of magnitude lower than the existing voltage-domain neurons.
  arXiv  Detail & Related papers  (2022-02-04T08:24:03Z)
• POPPINS : A Population-Based Digital Spiking Neuromorphic Processor with Integer Quadratic Integrate-and-Fire Neurons [50.591267188664666]
  We propose a population-based digital spiking neuromorphic processor in 180nm process technology with two hierarchy populations. The proposed approach enables the developments of biomimetic neuromorphic system and various low-power, and low-latency inference processing applications.
  arXiv  Detail & Related papers  (2022-01-19T09:26:34Z)
• Dynamic Neural Diversification: Path to Computationally Sustainable Neural Networks [68.8204255655161]
  Small neural networks with a constrained number of trainable parameters, can be suitable resource-efficient candidates for many simple tasks. We explore the diversity of the neurons within the hidden layer during the learning process. We analyze how the diversity of the neurons affects predictions of the model.
  arXiv  Detail & Related papers  (2021-09-20T15:12:16Z)
• SeReNe: Sensitivity based Regularization of Neurons for Structured Sparsity in Neural Networks [13.60023740064471]
  SeReNe is a method for learning sparse topologies with a structure. We define the sensitivity of a neuron as the variation of the network output. By including the neuron sensitivity in the cost function as a regularization term, we areable to prune neurons with low sensitivity.
  arXiv  Detail & Related papers  (2021-02-07T10:53:30Z)
• Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
  We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
  arXiv  Detail & Related papers  (2020-02-02T21:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.