Spatio-temporal Structure of Excitation and Inhibition Emerges in Spiking Neural Networks with and without Biologically Plausible Constraints

• URL: http://arxiv.org/abs/2407.18917v1
• Date: Sun, 7 Jul 2024 11:55:48 GMT
• Title: Spatio-temporal Structure of Excitation and Inhibition Emerges in Spiking Neural Networks with and without Biologically Plausible Constraints
• Authors: Balázs Mészáros, James Knight, Thomas Nowotny,
• Abstract summary: We present a Spiking Neural Network (SNN) model that incorporates learnable synaptic delays. We implement a dynamic pruning strategy that combines DEEP R for connection removal and RigL for connection. We observed that the reintroduction-temporal patterns of excitation and inhibition appeared in the more biologically plausible model as well.
• Score: 0.06752396542927405
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a Spiking Neural Network (SNN) model that incorporates learnable synaptic delays using Dilated Convolution with Learnable Spacings (DCLS). We train this model on the Raw Heidelberg Digits keyword spotting benchmark using Backpropagation Through Time with surrogate gradients. Analysing the spatio-temporal structure of synaptic interactions in the network we observe that after training excitation and inhibition are grouped together both in space and time. To further enhance the efficiency and biological realism of our model, we implemented a dynamic pruning strategy that combines DEEP R for connection removal and RigL for connection reintroduction, ensuring that the network maintains optimal connectivity throughout training. Additionally, we incorporated Dale's Principle, enforcing each neuron to be exclusively excitatory or inhibitory -- aligning our model closer to biological neural networks. We observed that, after training, the spatio-temporal patterns of excitation and inhibition appeared in the more biologically plausible model as well. Our research demonstrates the potential of integrating learnable delays, dynamic pruning, and biological constraints to develop efficient SNN models for temporal data processing. Furthermore, our results enhance the understanding of spatio-temporal dynamics in SNNs -- suggesting that the spatio-temporal features which emerge from training are robust to both pruning and rewiring processes -- providing a solid foundation for future work in neuromorphic computing applications.

Related papers

• Contrastive Learning in Memristor-based Neuromorphic Systems [55.11642177631929]
  Spiking neural networks have become an important family of neuron-based models that sidestep many of the key limitations facing modern-day backpropagation-trained deep networks. In this work, we design and investigate a proof-of-concept instantiation of contrastive-signal-dependent plasticity (CSDP), a neuromorphic form of forward-forward-based, backpropagation-free learning.
  arXiv  Detail & Related papers  (2024-09-17T04:48:45Z)
• Autaptic Synaptic Circuit Enhances Spatio-temporal Predictive Learning of Spiking Neural Networks [23.613277062707844]
  Spiking Neural Networks (SNNs) emulate the integrated-fire-leak mechanism found in biological neurons. Existing SNNs predominantly rely on the Integrate-and-Fire Leaky (LIF) model. This paper proposes a novel S-patioTemporal Circuit (STC) model.
  arXiv  Detail & Related papers  (2024-06-01T11:17:27Z)
• ELiSe: Efficient Learning of Sequences in Structured Recurrent Networks [1.5931140598271163]
  We build a model for efficient learning sequences using only local always-on and phase-free plasticity. We showcase the capabilities of ELiSe in a mock-up of birdsong learning, and demonstrate its flexibility with respect to parametrization.
  arXiv  Detail & Related papers  (2024-02-26T17:30:34Z)
• TC-LIF: A Two-Compartment Spiking Neuron Model for Long-Term Sequential Modelling [54.97005925277638]
  The identification of sensory cues associated with potential opportunities and dangers is frequently complicated by unrelated events that separate useful cues by long delays. It remains a challenging task for state-of-the-art spiking neural networks (SNNs) to establish long-term temporal dependency between distant cues. We propose a novel biologically inspired Two-Compartment Leaky Integrate-and-Fire spiking neuron model, dubbed TC-LIF.
  arXiv  Detail & Related papers  (2023-08-25T08:54:41Z)
• How neural networks learn to classify chaotic time series [77.34726150561087]
  We study the inner workings of neural networks trained to classify regular-versus-chaotic time series. We find that the relation between input periodicity and activation periodicity is key for the performance of LKCNN models.
  arXiv  Detail & Related papers  (2023-06-04T08:53:27Z)
• STSC-SNN: Spatio-Temporal Synaptic Connection with Temporal Convolution and Attention for Spiking Neural Networks [7.422913384086416]
  Spiking Neural Networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal processing capability. Existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither which can extract temporal dependencies adequately. We take inspiration from biological synapses and propose a synaptic connection SNN model, to enhance the synapse-temporal receptive fields of synaptic connections. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks.
  arXiv  Detail & Related papers  (2022-10-11T08:13:22Z)
• Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons [0.7340017786387767]
  We introduce Latent Equilibrium, a new framework for inference and learning in networks of slow components. We derive disentangled neuron and synapse dynamics from a prospective energy function. We show how our principle can be applied to detailed models of cortical microcircuitry.
  arXiv  Detail & Related papers  (2021-10-27T16:15:55Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)
• Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data [77.92736596690297]
  We introduce a novel training strategy that allows learning not only the input-output behavior of an RNN but also its internal network dynamics. We test the proposed method by training an RNN to simultaneously reproduce internal dynamics and output signals of a physiologically-inspired neural model. Remarkably, we show that the reproduction of the internal dynamics is successful even when the training algorithm relies on the activities of a small subset of neurons.
  arXiv  Detail & Related papers  (2020-05-05T14:16:54Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)
• Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network [7.503685643036081]
  A bio-plausible SNN model with spatial-temporal property is a complex dynamic system. We formulate SNN as a network of infinite impulse response (IIR) filters with neuron nonlinearity. We propose a training algorithm that is capable to learn spatial-temporal patterns by searching for the optimal synapse filter kernels and weights.
  arXiv  Detail & Related papers  (2020-02-19T01:27:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.