EventRPG: Event Data Augmentation with Relevance Propagation Guidance

• URL: http://arxiv.org/abs/2403.09274v1
• Date: Thu, 14 Mar 2024 10:52:45 GMT
• Title: EventRPG: Event Data Augmentation with Relevance Propagation Guidance
• Authors: Mingyuan Sun, Donghao Zhang, Zongyuan Ge, Jiaxu Wang, Jia Li, Zheng Fang, Renjing Xu,
• Abstract summary: Overfitting is a critical problem in event-based classification tasks for Spiking Neural Network (SNN) Data augmentation is a simple but efficient method to alleviate overfitting and improve the generalization ability of neural networks. We propose EventRPG, which leverages relevance propagation on the spiking neural network for more efficient augmentation.
• Score: 25.899827299880577
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event camera, a novel bio-inspired vision sensor, has drawn a lot of attention for its low latency, low power consumption, and high dynamic range. Currently, overfitting remains a critical problem in event-based classification tasks for Spiking Neural Network (SNN) due to its relatively weak spatial representation capability. Data augmentation is a simple but efficient method to alleviate overfitting and improve the generalization ability of neural networks, and saliency-based augmentation methods are proven to be effective in the image processing field. However, there is no approach available for extracting saliency maps from SNNs. Therefore, for the first time, we present Spiking Layer-Time-wise Relevance Propagation rule (SLTRP) and Spiking Layer-wise Relevance Propagation rule (SLRP) in order for SNN to generate stable and accurate CAMs and saliency maps. Based on this, we propose EventRPG, which leverages relevance propagation on the spiking neural network for more efficient augmentation. Our proposed method has been evaluated on several SNN structures, achieving state-of-the-art performance in object recognition tasks including N-Caltech101, CIFAR10-DVS, with accuracies of 85.62% and 85.55%, as well as action recognition task SL-Animals with an accuracy of 91.59%. Our code is available at https://github.com/myuansun/EventRPG.

Related papers

• Spiking Neural Network as Adaptive Event Stream Slicer [10.279359105384334]
  Event-based cameras provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information. We propose SpikeSlicer, a novel-designed plug-and-play event processing method capable of splitting events stream adaptively.
  arXiv  Detail & Related papers  (2024-10-03T06:41:10Z)
• Towards Low-latency Event-based Visual Recognition with Hybrid Step-wise Distillation Spiking Neural Networks [50.32980443749865]
  Spiking neural networks (SNNs) have garnered significant attention for their low power consumption and high biologicalability. Current SNNs struggle to balance accuracy and latency in neuromorphic datasets. We propose Step-wise Distillation (HSD) method, tailored for neuromorphic datasets.
  arXiv  Detail & Related papers  (2024-09-19T06:52:34Z)
• Hyperspectral Image Classification Based on Faster Residual Multi-branch Spiking Neural Network [6.166929138912052]
  This paper builds a spiking neural network (SNN) based on the leaky integrate-and-fire (LIF) neuron model for HSI classification tasks. SNN-SWMR requires a time step reduction of about 84%, training time, and testing time reduction of about 63% and 70% at the same accuracy.
  arXiv  Detail & Related papers  (2024-09-18T00:51:01Z)
• Training a General Spiking Neural Network with Improved Efficiency and Minimum Latency [4.503744528661997]
  Spiking Neural Networks (SNNs) operate in an event-driven manner and employ binary spike representation. This paper proposes a general training framework that enhances feature learning and activation efficiency within a limited time step.
  arXiv  Detail & Related papers  (2024-01-05T09:54:44Z)
• LC-TTFS: Towards Lossless Network Conversion for Spiking Neural Networks with TTFS Coding [55.64533786293656]
  We show that our algorithm can achieve a near-perfect mapping between the activation values of an ANN and the spike times of an SNN on a number of challenging AI tasks. The study paves the way for deploying ultra-low-power TTFS-based SNNs on power-constrained edge computing platforms.
  arXiv  Detail & Related papers  (2023-10-23T14:26:16Z)
• Gated Attention Coding for Training High-performance and Efficient Spiking Neural Networks [22.66931446718083]
  Gated Attention Coding (GAC) is a plug-and-play module that leverages the multi-dimensional attention unit to efficiently encode inputs into powerful representations. GAC functions as a preprocessing layer that does not disrupt the spike-driven nature of the SNN. Experiments on CIFAR10/100 and ImageNet datasets demonstrate that GAC achieves state-of-the-art accuracy with remarkable efficiency.
  arXiv  Detail & Related papers  (2023-08-12T14:42:02Z)
• Deep Multi-Threshold Spiking-UNet for Image Processing [51.88730892920031]
  This paper introduces the novel concept of Spiking-UNet for image processing, which combines the power of Spiking Neural Networks (SNNs) with the U-Net architecture. To achieve an efficient Spiking-UNet, we face two primary challenges: ensuring high-fidelity information propagation through the network via spikes and formulating an effective training strategy. Experimental results show that, on image segmentation and denoising, our Spiking-UNet achieves comparable performance to its non-spiking counterpart.
  arXiv  Detail & Related papers  (2023-07-20T16:00:19Z)
• Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation [70.75043144299168]
  Spiking Neural Network (SNN) is a promising energy-efficient AI model when implemented on neuromorphic hardware. It is a challenge to efficiently train SNNs due to their non-differentiability. We propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance.
  arXiv  Detail & Related papers  (2022-05-01T12:44:49Z)
• Hybrid SNN-ANN: Energy-Efficient Classification and Object Detection for Event-Based Vision [64.71260357476602]
  Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks. We propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection.
  arXiv  Detail & Related papers  (2021-12-06T23:45:58Z)
• Spiking Neural Networks for Visual Place Recognition via Weighted Neuronal Assignments [24.754429120321365]
  Spiking neural networks (SNNs) offer both compelling potential advantages, including energy efficiency and low latencies. One promising area for high performance SNNs is template matching and image recognition. This research introduces the first high performance SNN for the Visual Place Recognition (VPR) task.
  arXiv  Detail & Related papers  (2021-09-14T05:40:40Z)
• Progressive Tandem Learning for Pattern Recognition with Deep Spiking Neural Networks [80.15411508088522]
  Spiking neural networks (SNNs) have shown advantages over traditional artificial neural networks (ANNs) for low latency and high computational efficiency. We propose a novel ANN-to-SNN conversion and layer-wise learning framework for rapid and efficient pattern recognition.
  arXiv  Detail & Related papers  (2020-07-02T15:38:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.