ARNN: Attentive Recurrent Neural Network for Multi-channel EEG Signals
to Identify Epileptic Seizures
- URL: http://arxiv.org/abs/2403.03276v1
- Date: Tue, 5 Mar 2024 19:15:17 GMT
- Title: ARNN: Attentive Recurrent Neural Network for Multi-channel EEG Signals
to Identify Epileptic Seizures
- Authors: Salim Rukhsar and Anil Kumar Tiwari
- Abstract summary: We propose an Attentive Recurrent Neural Network (ARNN), which recurrently applies attention layers along a sequence.
The proposed model operates on multi-channel EEG signals rather than single channel signals and leverages parallel computation.
- Score: 2.8244056068360095
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We proposed an Attentive Recurrent Neural Network (ARNN), which recurrently
applies attention layers along a sequence and has linear complexity with
respect to the sequence length. The proposed model operates on multi-channel
EEG signals rather than single channel signals and leverages parallel
computation. In this cell, the attention layer is a computational unit that
efficiently applies self-attention and cross-attention mechanisms to compute a
recurrent function over a wide number of state vectors and input signals. Our
architecture is inspired in part by the attention layer and long short-term
memory (LSTM) cells, and it uses long-short style gates, but it scales this
typical cell up by several orders to parallelize for multi-channel EEG signals.
It inherits the advantages of attention layers and LSTM gate while avoiding
their respective drawbacks. We evaluated the model effectiveness through
extensive experiments with heterogeneous datasets, including the CHB-MIT and
UPenn and Mayos Clinic, CHB-MIT datasets. The empirical findings suggest that
the ARNN model outperforms baseline methods such as LSTM, Vision Transformer
(ViT), Compact Convolution Transformer (CCT), and R-Transformer (RT),
showcasing superior performance and faster processing capabilities across a
wide range of tasks. The code has been made publicly accessible at
\url{https://github.com/Salim-Lysiun/ARNN}.
Related papers
- TBSN: Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising [94.09442506816724]
Blind-spot networks (BSN) have been prevalent network architectures in self-supervised image denoising (SSID)
We present a transformer-based blind-spot network (TBSN) by analyzing and redesigning the transformer operators that meet the blind-spot requirement.
For spatial self-attention, an elaborate mask is applied to the attention matrix to restrict its receptive field, thus mimicking the dilated convolution.
For channel self-attention, we observe that it may leak the blind-spot information when the channel number is greater than spatial size in the deep layers of multi-scale architectures.
arXiv Detail & Related papers (2024-04-11T15:39:10Z) - RetMIL: Retentive Multiple Instance Learning for Histopathological Whole Slide Image Classification [10.365234803533982]
We propose a retentive MIL method called RetMIL, which processes WSI sequences through hierarchical feature propagation structure.
At the local level, the WSI sequence is divided into multiple subsequences. Tokens of each subsequence are updated through a parallel linear retention mechanism.
At the global level, subsequences are fused into a global sequence, then updated through a serial retention mechanism, and finally the slide-level representation is obtained through a global attention pooling.
arXiv Detail & Related papers (2024-03-16T08:50:47Z) - Heterogenous Memory Augmented Neural Networks [84.29338268789684]
We introduce a novel heterogeneous memory augmentation approach for neural networks.
By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead.
We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
arXiv Detail & Related papers (2023-10-17T01:05:28Z) - Joint Channel Estimation and Feedback with Masked Token Transformers in
Massive MIMO Systems [74.52117784544758]
This paper proposes an encoder-decoder based network that unveils the intrinsic frequency-domain correlation within the CSI matrix.
The entire encoder-decoder network is utilized for channel compression.
Our method outperforms state-of-the-art channel estimation and feedback techniques in joint tasks.
arXiv Detail & Related papers (2023-06-08T06:15:17Z) - Online Transformers with Spiking Neurons for Fast Prosthetic Hand
Control [1.6114012813668934]
In this paper, instead of the self-attention mechanism, we use a sliding window attention mechanism.
We show that this mechanism is more efficient for continuous signals with finite-range dependencies between input and target.
Our results hold great promises for accurate and fast online processing of sEMG signals for smooth prosthetic hand control.
arXiv Detail & Related papers (2023-03-21T13:59:35Z) - Towards Energy-Efficient, Low-Latency and Accurate Spiking LSTMs [1.7969777786551424]
Spiking Neural Networks (SNNs) have emerged as an attractive-temporal computing paradigm vision for complex tasks.
We propose an optimized spiking long short-term memory networks (LSTM) training framework that involves a novel.
rev-to-SNN conversion framework, followed by SNN training.
We evaluate our framework on sequential learning tasks including temporal M, Google Speech Commands (GSC) datasets, and UCI Smartphone on different LSTM architectures.
arXiv Detail & Related papers (2022-10-23T04:10:27Z) - Image Classification using Sequence of Pixels [3.04585143845864]
This study compares sequential image classification methods based on recurrent neural networks.
We describe methods based on Long-Short-Term memory(LSTM), bidirectional Long-Short-Term memory(BiLSTM) architectures, etc.
arXiv Detail & Related papers (2022-09-23T09:42:44Z) - Working Memory Connections for LSTM [51.742526187978726]
We show that Working Memory Connections constantly improve the performance of LSTMs on a variety of tasks.
Numerical results suggest that the cell state contains useful information that is worth including in the gate structure.
arXiv Detail & Related papers (2021-08-31T18:01:30Z) - Learning to Estimate RIS-Aided mmWave Channels [50.15279409856091]
We focus on uplink cascaded channel estimation, where known and fixed base station combining and RIS phase control matrices are considered for collecting observations.
To boost the estimation performance and reduce the training overhead, the inherent channel sparsity of mmWave channels is leveraged in the deep unfolding method.
It is verified that the proposed deep unfolding network architecture can outperform the least squares (LS) method with a relatively smaller training overhead and online computational complexity.
arXiv Detail & Related papers (2021-07-27T06:57:56Z) - PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive
Learning [109.84770951839289]
We present PredRNN, a new recurrent network for learning visual dynamics from historical context.
We show that our approach obtains highly competitive results on three standard datasets.
arXiv Detail & Related papers (2021-03-17T08:28:30Z) - Automatic Remaining Useful Life Estimation Framework with Embedded
Convolutional LSTM as the Backbone [5.927250637620123]
We propose a new LSTM variant called embedded convolutional LSTM (E NeuralTM)
In ETM a group of different 1D convolutions is embedded into the LSTM structure. Through this, the temporal information is preserved between and within windows.
We show the superiority of our proposed ETM approach over the state-of-the-art approaches on several widely used benchmark data sets for RUL Estimation.
arXiv Detail & Related papers (2020-08-10T08:34:20Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.