Feature Attribution Explanations for Spiking Neural Networks

• URL: http://arxiv.org/abs/2311.02110v1
• Date: Thu, 2 Nov 2023 21:30:56 GMT
• Title: Feature Attribution Explanations for Spiking Neural Networks
• Abstract summary: Third-generation artificial neural networks, Spiking Neural Networks (SNNs), can be efficiently implemented on hardware. Their implementation on neuromorphic chips opens a broad range of applications, such as machine learning-based autonomous control and intelligent biomedical devices. We present textitTemporal Spike Attribution (TSA), a local explanation method for SNNs.
• Score: 3.5824464725360077
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Third-generation artificial neural networks, Spiking Neural Networks (SNNs), can be efficiently implemented on hardware. Their implementation on neuromorphic chips opens a broad range of applications, such as machine learning-based autonomous control and intelligent biomedical devices. In critical applications, however, insight into the reasoning of SNNs is important, thus SNNs need to be equipped with the ability to explain how decisions are reached. We present \textit{Temporal Spike Attribution} (TSA), a local explanation method for SNNs. To compute the explanation, we aggregate all information available in model-internal variables: spike times and model weights. We evaluate TSA on artificial and real-world time series data and measure explanation quality w.r.t. multiple quantitative criteria. We find that TSA correctly identifies a small subset of input features relevant to the decision (i.e., is output-complete and compact) and generates similar explanations for similar inputs (i.e., is continuous). Further, our experiments show that incorporating the notion of \emph{absent} spikes improves explanation quality. Our work can serve as a starting point for explainable SNNs, with future implementations on hardware yielding not only predictions but also explanations in a broad range of application scenarios. Source code is available at https://github.com/ElisaNguyen/tsa-explanations.

Related papers

• Binary Spiking Neural Networks as Causal Models [13.300821692861271]
  We provide a causal analysis of Binary Spiking Neural Networks (BSNNs) to explain their behavior.<n>We are able to explain the output of the network by leveraging logic-based methods.
  arXiv  Detail & Related papers  (2026-04-29T09:09:48Z)
• xDNN(ASP): Explanation Generation System for Deep Neural Networks powered by Answer Set Programming [1.1936305787902912]
  xDNN(ASP) is an explanation generation system for deep neural networks that provides global explanations.<n>We show that xDNN(ASP) maintains a high-level of accuracy in the prediction task.
  arXiv  Detail & Related papers  (2026-01-07T12:08:00Z)
• S$^2$NN: Sub-bit Spiking Neural Networks [53.08060832135342]
  Spiking Neural Networks (SNNs) offer an energy-efficient paradigm for machine intelligence.<n>Despite recent advances in binary SNNs, the storage and computational demands remain substantial for large-scale networks.<n>We propose Sub-bit Spiking Neural Networks (S$2$NNs) that represent weights with less than one bit.
  arXiv  Detail & Related papers  (2025-09-29T04:17:44Z)
• Unveiling the Power of Sparse Neural Networks for Feature Selection [60.50319755984697]
  Sparse Neural Networks (SNNs) have emerged as powerful tools for efficient feature selection. We show that SNNs trained with dynamic sparse training (DST) algorithms can achieve, on average, more than $50%$ memory and $55%$ FLOPs reduction. Our findings show that feature selection with SNNs trained with DST algorithms can achieve, on average, more than $50%$ memory and $55%$ FLOPs reduction.
  arXiv  Detail & Related papers  (2024-08-08T16:48:33Z)
• Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning [91.29876772547348]
  Spiking neural networks (SNNs) are investigated as biologically inspired models of neural computation. This paper reveals that SNNs, when amalgamated with synaptic delay and temporal coding, are proficient in executing (knowledge) graph reasoning.
  arXiv  Detail & Related papers  (2024-05-27T05:53:30Z)
• An Automata-Theoretic Approach to Synthesizing Binarized Neural Networks [13.271286153792058]
  Quantized neural networks (QNNs) have been developed, with binarized neural networks (BNNs) restricted to binary values as a special case. This paper presents an automata-theoretic approach to synthesizing BNNs that meet designated properties.
  arXiv  Detail & Related papers  (2023-07-29T06:27:28Z)
• A Spiking Neural Network Structure Implementing Reinforcement Learning [0.0]
  In the present paper, I describe an SNN structure which, seemingly, can be used in wide range of reinforcement learning tasks. The SNN structure considered in the paper includes spiking neurons described by a generalization of the LIFAT (leaky integrate-and-fire neuron with adaptive threshold) model. My concept is based on very general assumptions about RL task characteristics and has no visible limitations on its applicability.
  arXiv  Detail & Related papers  (2022-04-09T09:08:10Z)
• NeuroView-RNN: It's About Time [25.668977252138905]
  A key interpretability issue with RNNs is that it is not clear how each hidden state per time step contributes to the decision-making process. We propose NeuroView-RNN as a family of new RNN architectures that explains how all the time steps are used for the decision-making process. We showcase the benefits of NeuroView-RNN by evaluating on a multitude of diverse time-series datasets.
  arXiv  Detail & Related papers  (2022-02-23T22:29:11Z)
• Task-Agnostic Graph Explanations [50.17442349253348]
  Graph Neural Networks (GNNs) have emerged as powerful tools to encode graph structured data. Existing learning-based GNN explanation approaches are task-specific in training. We propose a Task-Agnostic GNN Explainer (TAGE) trained under self-supervision with no knowledge of downstream tasks.
  arXiv  Detail & Related papers  (2022-02-16T21:11:47Z)
• SpikeMS: Deep Spiking Neural Network for Motion Segmentation [7.491944503744111]
  textitSpikeMS is the first deep encoder-decoder SNN architecture for the real-world large-scale problem of motion segmentation. We show that textitSpikeMS is capable of textitincremental predictions, or predictions from smaller amounts of test data than it is trained on.
  arXiv  Detail & Related papers  (2021-05-13T21:34:55Z)
• Deep Neural Networks to Recover Unknown Physical Parameters from Oscillating Time Series [3.7277730514654555]
  We use deep neural networks to disentangle components of oscillating time series. We show that our architecture could prove useful for time-series processing in physics, when prior knowledge is incomplete.
  arXiv  Detail & Related papers  (2021-01-11T12:39:50Z)
• Boosting Deep Neural Networks with Geometrical Prior Knowledge: A Survey [77.99182201815763]
  Deep Neural Networks (DNNs) achieve state-of-the-art results in many different problem settings. DNNs are often treated as black box systems, which complicates their evaluation and validation. One promising field, inspired by the success of convolutional neural networks (CNNs) in computer vision tasks, is to incorporate knowledge about symmetric geometrical transformations.
  arXiv  Detail & Related papers  (2020-06-30T14:56:05Z)
• Neural Additive Models: Interpretable Machine Learning with Neural Nets [77.66871378302774]
  Deep neural networks (DNNs) are powerful black-box predictors that have achieved impressive performance on a wide variety of tasks. We propose Neural Additive Models (NAMs) which combine some of the expressivity of DNNs with the inherent intelligibility of generalized additive models. NAMs learn a linear combination of neural networks that each attend to a single input feature.
  arXiv  Detail & Related papers  (2020-04-29T01:28:32Z)
• Architecture Disentanglement for Deep Neural Networks [174.16176919145377]
  We introduce neural architecture disentanglement (NAD) to explain the inner workings of deep neural networks (DNNs) NAD learns to disentangle a pre-trained DNN into sub-architectures according to independent tasks, forming information flows that describe the inference processes. Results show that misclassified images have a high probability of being assigned to task sub-architectures similar to the correct ones.
  arXiv  Detail & Related papers  (2020-03-30T08:34:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.