Catwalk: Unary Top-K for Efficient Ramp-No-Leak Neuron Design for Temporal Neural Networks

• URL: http://arxiv.org/abs/2508.21267v1
• Date: Thu, 28 Aug 2025 23:50:36 GMT
• Title: Catwalk: Unary Top-K for Efficient Ramp-No-Leak Neuron Design for Temporal Neural Networks
• Authors: Devon Lister, Prabhu Vellaisamy, John Paul Shen, Di Wu,
• Abstract summary: We propose a Catwalk neuron implementation by relocating spikes in a spike volley as a sorted subset cluster via unary top-k.<n>Catwalk is 1.39x and 1.86x better in area and power, respectively, as compared to existing0-RNL neurons.
• Score: 3.0670569650183928
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal neural networks (TNNs) are neuromorphic neural networks that utilize bit-serial temporal coding. TNNs are composed of columns, which in turn employ neurons as their building blocks. Each neuron processes volleys of input spikes, modulated by associated synaptic weights, on its dendritic inputs. Recently proposed neuron implementation in CMOS employs a Spike Response Model (SRM) with a ramp-no-leak (RNL) response function and assumes all the inputs can carry spikes. However, in actual spike volleys, only a small subset of the dendritic inputs actually carry spikes in each compute cycle. This form of sparsity can be exploited to achieve better hardware efficiency. In this paper, we propose a Catwalk neuron implementation by relocating spikes in a spike volley as a sorted subset cluster via unary top-k. Such relocation can significantly reduce the cost of the subsequent parallel counter (PC) for accumulating the response functions from the spiking inputs. This can lead to improvements on area and power efficiency in RNL neuron implementation. Place-and-route results show Catwalk is 1.39x and 1.86x better in area and power, respectively, as compared to existing SRM0-RNL neurons.

Related papers

• Language Model Circuits Are Sparse in the Neuron Basis [50.460651620833055]
  We show that textbfMLP neurons are as sparse a feature basis as SAEs.<n>This work advances automated interpretability of language models without additional training costs.
  arXiv  Detail & Related papers  (2026-01-30T05:41:19Z)
• Multiplication-Free Parallelizable Spiking Neurons with Efficient Spatio-Temporal Dynamics [40.43988645674521]
  Spiking Neural Networks (SNNs) are distinguished from Artificial Neural Networks (ANNs) for their complex neuronal dynamics and sparse binary activations (spikes) inspired by the biological neural system.<n>Traditional neuron models use iterative step-by-step dynamics, resulting in serial computation and slow training speed of SNNs.<n>Recently, parallelizable spiking neuron models have been proposed to fully utilize the massive parallel computing ability of graphics processing units to accelerate the training of SNNs.
  arXiv  Detail & Related papers  (2025-01-24T13:44:08Z)
• Fully Spiking Actor Network with Intra-layer Connections for Reinforcement Learning [51.386945803485084]
  We focus on the task where the agent needs to learn multi-dimensional deterministic policies to control. Most existing spike-based RL methods take the firing rate as the output of SNNs, and convert it to represent continuous action space (i.e., the deterministic policy) through a fully-connected layer. To develop a fully spiking actor network without any floating-point matrix operations, we draw inspiration from the non-spiking interneurons found in insects.
  arXiv  Detail & Related papers  (2024-01-09T07:31:34Z)
• An On-Chip Trainable Neuron Circuit for SFQ-Based Spiking Neural Networks [4.825037489691159]
  We present an on-chip trainable neuron circuit for training spiking neural networks (SNN) Our proposed circuit suits bio-inspired spike-based time-dependent data computation for training spiking neural networks (SNN) The circuits are designed and optimized for the MIT LLQ5ee fabrication process.
  arXiv  Detail & Related papers  (2023-10-11T19:04:33Z)
• Co-learning synaptic delays, weights and adaptation in spiking neural networks [0.0]
  Spiking neural networks (SNN) distinguish themselves from artificial neural networks (ANN) because of their inherent temporal processing and spike-based computations. We show that data processing with spiking neurons can be enhanced by co-learning the connection weights with two other biologically inspired neuronal features.
  arXiv  Detail & Related papers  (2023-09-12T09:13:26Z)
• Synaptic Stripping: How Pruning Can Bring Dead Neurons Back To Life [0.0]
  We introduce Synaptic Stripping as a means to combat the dead neuron problem. By automatically removing problematic connections during training, we can regenerate dead neurons. We conduct several ablation studies to investigate these dynamics as a function of network width and depth.
  arXiv  Detail & Related papers  (2023-02-11T23:55:50Z)
• Desire Backpropagation: A Lightweight Training Algorithm for Multi-Layer Spiking Neural Networks based on Spike-Timing-Dependent Plasticity [13.384228628766236]
  Spiking neural networks (SNNs) are a viable alternative to conventional artificial neural networks. We present desire backpropagation, a method to derive the desired spike activity of all neurons, including the hidden ones. We trained three-layer networks to classify MNIST and Fashion-MNIST images and reached an accuracy of 98.41% and 87.56%, respectively.
  arXiv  Detail & Related papers  (2022-11-10T08:32:13Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• A Spiking Neural Network Structure Implementing Reinforcement Learning [0.0]
  In the present paper, I describe an SNN structure which, seemingly, can be used in wide range of reinforcement learning tasks. The SNN structure considered in the paper includes spiking neurons described by a generalization of the LIFAT (leaky integrate-and-fire neuron with adaptive threshold) model. My concept is based on very general assumptions about RL task characteristics and has no visible limitations on its applicability.
  arXiv  Detail & Related papers  (2022-04-09T09:08:10Z)
• Event-based Video Reconstruction via Potential-assisted Spiking Neural Network [48.88510552931186]
  Bio-inspired neural networks can potentially lead to greater computational efficiency on event-driven hardware. We propose a novel Event-based Video reconstruction framework based on a fully Spiking Neural Network (EVSNN) We find that the spiking neurons have the potential to store useful temporal information (memory) to complete such time-dependent tasks.
  arXiv  Detail & Related papers  (2022-01-25T02:05:20Z)
• And/or trade-off in artificial neurons: impact on adversarial robustness [91.3755431537592]
  Presence of sufficient number of OR-like neurons in a network can lead to classification brittleness and increased vulnerability to adversarial attacks. We define AND-like neurons and propose measures to increase their proportion in the network. Experimental results on the MNIST dataset suggest that our approach holds promise as a direction for further exploration.
  arXiv  Detail & Related papers  (2021-02-15T08:19:05Z)
• Non-linear Neurons with Human-like Apical Dendrite Activations [81.18416067005538]
  We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. We conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing.
  arXiv  Detail & Related papers  (2020-02-02T21:09:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.