Decoding ECoG signal into 3D hand translation using deep learning

• URL: http://arxiv.org/abs/2110.03528v1
• Date: Tue, 5 Oct 2021 15:41:04 GMT
• Title: Decoding ECoG signal into 3D hand translation using deep learning
• Authors: Maciej \'Sliwowski, Matthieu Martin, Antoine Souloumiac, Pierre Blanchart, Tetiana Aksenova
• Abstract summary: Motor brain-computer interfaces (BCIs) are promising technology that may enable motor-impaired people to interact with their environment. Most ECoG signal decoders used to predict continuous hand movements are linear models. Deep learning models, which are state-of-the-art in many problems, could be a solution to better capture this relationship.
• Score: 3.20238141000059
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Motor brain-computer interfaces (BCIs) are a promising technology that may enable motor-impaired people to interact with their environment. Designing real-time and accurate BCI is crucial to make such devices useful, safe, and easy to use by patients in a real-life environment. Electrocorticography (ECoG)-based BCIs emerge as a good compromise between invasiveness of the recording device and good spatial and temporal resolution of the recorded signal. However, most ECoG signal decoders used to predict continuous hand movements are linear models. These models have a limited representational capacity and may fail to capture the relationship between ECoG signal and continuous hand movements. Deep learning (DL) models, which are state-of-the-art in many problems, could be a solution to better capture this relationship. In this study, we tested several DL-based architectures to predict imagined 3D continuous hand translation using time-frequency features extracted from ECoG signals. The dataset used in the analysis is a part of a long-term clinical trial (ClinicalTrials.gov identifier: NCT02550522) and was acquired during a closed-loop experiment with a tetraplegic subject. The proposed architectures include multilayer perceptron (MLP), convolutional neural networks (CNN), and long short-term memory networks (LSTM). The accuracy of the DL-based and multilinear models was compared offline using cosine similarity. Our results show that CNN-based architectures outperform the current state-of-the-art multilinear model. The best architecture exploited the spatial correlation between neighboring electrodes with CNN and benefited from the sequential character of the desired hand trajectory by using LSTMs. Overall, DL increased the average cosine similarity, compared to the multilinear model, by up to 60%, from 0.189 to 0.302 and from 0.157 to 0.249 for the left and right hand, respectively.

Related papers

• 3D-CLMI: A Motor Imagery EEG Classification Model via Fusion of 3D-CNN and LSTM with Attention [0.174048653626208]
  This paper proposed a model that combined a three-dimensional convolutional neural network (CNN) with a long short-term memory (LSTM) network to classify motor imagery (MI) signals. Experimental results showed that this model achieved a classification accuracy of 92.7% and an F1-score of 0.91 on the public dataset BCI Competition IV dataset 2a. The model greatly improved the classification accuracy of users' motor imagery intentions, giving brain-computer interfaces better application prospects in emerging fields such as autonomous vehicles and medical rehabilitation.
  arXiv  Detail & Related papers  (2023-12-20T03:38:24Z)
• Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unit [56.801856519460465]
  Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations. We demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting.
  arXiv  Detail & Related papers  (2023-04-14T09:39:06Z)
• Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control [1.6114012813668934]
  In this paper, instead of the self-attention mechanism, we use a sliding window attention mechanism. We show that this mechanism is more efficient for continuous signals with finite-range dependencies between input and target. Our results hold great promises for accurate and fast online processing of sEMG signals for smooth prosthetic hand control.
  arXiv  Detail & Related papers  (2023-03-21T13:59:35Z)
• Deep learning for ECoG brain-computer interface: end-to-end vs. hand-crafted features [4.7773230870500605]
  Brain signals are temporal data with a low signal-to-noise ratio, uncertain labels, and nonstationary data in time. These factors may influence the training process and slow down the models' performance improvement. This paper compares models that use raw ECoG signal and time-frequency features for BCI motor imagery decoding.
  arXiv  Detail & Related papers  (2022-10-05T20:18:30Z)
• NAF: Neural Attenuation Fields for Sparse-View CBCT Reconstruction [79.13750275141139]
  This paper proposes a novel and fast self-supervised solution for sparse-view CBCT reconstruction. The desired attenuation coefficients are represented as a continuous function of 3D spatial coordinates, parameterized by a fully-connected deep neural network. A learning-based encoder entailing hash coding is adopted to help the network capture high-frequency details.
  arXiv  Detail & Related papers  (2022-09-29T04:06:00Z)
• An intertwined neural network model for EEG classification in brain-computer interfaces [0.6696153817334769]
  The brain computer interface (BCI) is a nonstimulatory direct and occasionally bidirectional communication link between the brain and a computer or an external device. We present a deep neural network architecture specifically engineered to provide state-of-the-art performance in multiclass motor imagery classification.
  arXiv  Detail & Related papers  (2022-08-04T09:00:34Z)
• Learning A 3D-CNN and Transformer Prior for Hyperspectral Image Super-Resolution [80.93870349019332]
  We propose a novel HSISR method that uses Transformer instead of CNN to learn the prior of HSIs. Specifically, we first use the gradient algorithm to solve the HSISR model, and then use an unfolding network to simulate the iterative solution processes.
  arXiv  Detail & Related papers  (2021-11-27T15:38:57Z)
• Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction [92.16318571149553]
  We propose a multiscale-temporal graph neural network (MST-GNN) to predict the future 3D-based skeleton human poses. The MST-GNN outperforms state-of-the-art methods in both short and long-term motion prediction.
  arXiv  Detail & Related papers  (2021-08-25T14:05:37Z)
• Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data with Spatial Information [52.635997570873194]
  This work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to process complex multi-dimensional time series data with spatial information. The proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
  arXiv  Detail & Related papers  (2021-01-12T20:08:18Z)
• GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals [8.19994663278877]
  A novel deep learning framework based on the graph convolutional neural networks (GCNs) is presented to enhance the decoding performance of raw EEG signals. The introduced approach has been shown to converge for both personalized and group-wise predictions.
  arXiv  Detail & Related papers  (2020-06-16T04:57:12Z)
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks [69.25956542388653]
  Deep learning (DL) algorithms are gaining weight in academic and industrial settings. We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework. The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
  arXiv  Detail & Related papers  (2020-05-11T16:29:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.