Enhancing Adversarial Robustness in SNNs with Sparse Gradients

• URL: http://arxiv.org/abs/2405.20355v1
• Date: Thu, 30 May 2024 05:39:27 GMT
• Title: Enhancing Adversarial Robustness in SNNs with Sparse Gradients
• Authors: Yujia Liu, Tong Bu, Jianhao Ding, Zecheng Hao, Tiejun Huang, Zhaofei Yu,
• Abstract summary: Spiking Neural Networks (SNNs) have attracted great attention for their energy-efficient operations and biologically inspired structures. Existing techniques, whether adapted from ANNs or specifically designed for SNNs, exhibit limitations in training SNNs or defending against strong attacks. We propose a novel approach to enhance the robustness of SNNs through gradient sparsity regularization.
• Score: 46.15229142258264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spiking Neural Networks (SNNs) have attracted great attention for their energy-efficient operations and biologically inspired structures, offering potential advantages over Artificial Neural Networks (ANNs) in terms of energy efficiency and interpretability. Nonetheless, similar to ANNs, the robustness of SNNs remains a challenge, especially when facing adversarial attacks. Existing techniques, whether adapted from ANNs or specifically designed for SNNs, exhibit limitations in training SNNs or defending against strong attacks. In this paper, we propose a novel approach to enhance the robustness of SNNs through gradient sparsity regularization. We observe that SNNs exhibit greater resilience to random perturbations compared to adversarial perturbations, even at larger scales. Motivated by this, we aim to narrow the gap between SNNs under adversarial and random perturbations, thereby improving their overall robustness. To achieve this, we theoretically prove that this performance gap is upper bounded by the gradient sparsity of the probability associated with the true label concerning the input image, laying the groundwork for a practical strategy to train robust SNNs by regularizing the gradient sparsity. We validate the effectiveness of our approach through extensive experiments on both image-based and event-based datasets. The results demonstrate notable improvements in the robustness of SNNs. Our work highlights the importance of gradient sparsity in SNNs and its role in enhancing robustness.

Related papers

• RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Coding [17.342181435229573]
  Spiking Neural Networks (SNNs) have received widespread attention due to their unique neuronal dynamics and low-power nature. Previous research empirically shows that SNNs with Poisson coding are more robust than Artificial Neural Networks (ANNs) on small-scale datasets. This work theoretically demonstrates that SNN's inherent adversarial robustness stems from its Poisson coding.
  arXiv  Detail & Related papers  (2024-07-29T15:26:15Z)
• Adversarially Robust Spiking Neural Networks Through Conversion [16.2319630026996]
  Spiking neural networks (SNNs) provide an energy-efficient alternative to a variety of artificial neural network (ANN) based AI applications. As the progress in neuromorphic computing with SNNs expands their use in applications, the problem of adversarial robustness of SNNs becomes more pronounced.
  arXiv  Detail & Related papers  (2023-11-15T08:33:46Z)
• Inherent Redundancy in Spiking Neural Networks [24.114844269113746]
  Spiking Networks (SNNs) are a promising energy-efficient alternative to conventional artificial neural networks. In this work, we focus on three key questions regarding inherent redundancy in SNNs. We propose an Advance Attention (ASA) module to harness SNNs' redundancy.
  arXiv  Detail & Related papers  (2023-08-16T08:58:25Z)
• On the Intrinsic Structures of Spiking Neural Networks [66.57589494713515]
  Recent years have emerged a surge of interest in SNNs owing to their remarkable potential to handle time-dependent and event-driven data. There has been a dearth of comprehensive studies examining the impact of intrinsic structures within spiking computations. This work delves deep into the intrinsic structures of SNNs, by elucidating their influence on the expressivity of SNNs.
  arXiv  Detail & Related papers  (2022-06-21T09:42:30Z)
• Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars [4.184276171116354]
  Spiking Neural Networks (SNNs) have emerged as the low-power alternative to Artificial Neural Networks (ANNs) We study the effect of crossbar non-idealities and intrinsicity on the performance of SNNs.
  arXiv  Detail & Related papers  (2022-06-20T07:07:41Z)
• Trustworthy Graph Neural Networks: Aspects, Methods and Trends [115.84291569988748]
  Graph neural networks (GNNs) have emerged as competent graph learning methods for diverse real-world scenarios. Performance-oriented GNNs have exhibited potential adverse effects like vulnerability to adversarial attacks. To avoid these unintentional harms, it is necessary to build competent GNNs characterised by trustworthiness.
  arXiv  Detail & Related papers  (2022-05-16T02:21:09Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Toward Robust Spiking Neural Network Against Adversarial Perturbation [22.56553160359798]
  spiking neural networks (SNNs) are deployed increasingly in real-world efficiency critical applications. Researchers have already demonstrated an SNN can be attacked with adversarial examples. To the best of our knowledge, this is the first analysis on robust training of SNNs.
  arXiv  Detail & Related papers  (2022-04-12T21:26:49Z)
• Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks [64.23331120621118]
  We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs) INNs are a class of implicit learning models that use implicit equations as layers. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs.
  arXiv  Detail & Related papers  (2022-04-01T03:31:27Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.