Generating EEG features from Acoustic features

• URL: http://arxiv.org/abs/2003.00007v2
• Date: Thu, 19 Mar 2020 01:33:53 GMT
• Title: Generating EEG features from Acoustic features
• Authors: Gautam Krishna, Co Tran, Mason Carnahan, Yan Han, Ahmed H Tewfik
• Abstract summary: We use recurrent neural network (RNN) based regression model and generative adversarial network (GAN) to predict EEG features from acoustic features. We compare our results with the previously studied problem on speech synthesis using EEG.
• Score: 13.089515271477824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we demonstrate predicting electroencephalograpgy (EEG) features from acoustic features using recurrent neural network (RNN) based regression model and generative adversarial network (GAN). We predict various types of EEG features from acoustic features. We compare our results with the previously studied problem on speech synthesis using EEG and our results demonstrate that EEG features can be generated from acoustic features with lower root mean square error (RMSE), normalized RMSE values compared to generating acoustic features from EEG features (ie: speech synthesis using EEG) when tested using the same data sets.

Related papers

• DGSD: Dynamical Graph Self-Distillation for EEG-Based Auditory Spatial Attention Detection [49.196182908826565]
  Auditory Attention Detection (AAD) aims to detect target speaker from brain signals in a multi-speaker environment. Current approaches primarily rely on traditional convolutional neural network designed for processing Euclidean data like images. This paper proposes a dynamical graph self-distillation (DGSD) approach for AAD, which does not require speech stimuli as input.
  arXiv  Detail & Related papers  (2023-09-07T13:43:46Z)
• Task-oriented Self-supervised Learning for Anomaly Detection in Electroencephalography [51.45515911920534]
  A task-oriented self-supervised learning approach is proposed to train a more effective anomaly detector. A specific two branch convolutional neural network with larger kernels is designed as the feature extractor. The effectively designed and trained feature extractor has shown to be able to extract better feature representations from EEGs.
  arXiv  Detail & Related papers  (2022-07-04T13:15:08Z)
• SOMOS: The Samsung Open MOS Dataset for the Evaluation of Neural Text-to-Speech Synthesis [50.236929707024245]
  The SOMOS dataset is the first large-scale mean opinion scores (MOS) dataset consisting of solely neural text-to-speech (TTS) samples. It consists of 20K synthetic utterances of the LJ Speech voice, a public domain speech dataset.
  arXiv  Detail & Related papers  (2022-04-06T18:45:20Z)
• Exploiting Cross Domain Acoustic-to-articulatory Inverted Features For Disordered Speech Recognition [57.15942628305797]
  Articulatory features are invariant to acoustic signal distortion and have been successfully incorporated into automatic speech recognition systems for normal speech. This paper presents a cross-domain acoustic-to-articulatory (A2A) inversion approach that utilizes the parallel acoustic-articulatory data of the 15-hour TORGO corpus in model training. Cross-domain adapted to the 102.7-hour UASpeech corpus and to produce articulatory features.
  arXiv  Detail & Related papers  (2022-03-19T08:47:18Z)
• EEGminer: Discovering Interpretable Features of Brain Activity with Learnable Filters [72.19032452642728]
  We propose a novel differentiable EEG decoding pipeline consisting of learnable filters and a pre-determined feature extraction module. We demonstrate the utility of our model towards emotion recognition from EEG signals on the SEED dataset and on a new EEG dataset of unprecedented size. The discovered features align with previous neuroscience studies and offer new insights, such as marked differences in the functional connectivity profile between left and right temporal areas during music listening.
  arXiv  Detail & Related papers  (2021-10-19T14:22:04Z)
• A Novel Transferability Attention Neural Network Model for EEG Emotion Recognition [51.203579838210885]
  We propose a transferable attention neural network (TANN) for EEG emotion recognition. TANN learns the emotional discriminative information by highlighting the transferable EEG brain regions data and samples adaptively. This can be implemented by measuring the outputs of multiple brain-region-level discriminators and one single sample-level discriminator.
  arXiv  Detail & Related papers  (2020-09-21T02:42:30Z)
• Constrained Variational Autoencoder for improving EEG based Speech Recognition Systems [3.5786621294068377]
  We introduce a recurrent neural network (RNN) based variational autoencoder (VAE) model with a new constrained loss function. We demonstrate that both continuous and isolated speech recognition systems trained and tested using EEG features generated from raw EEG features.
  arXiv  Detail & Related papers  (2020-06-01T06:03:50Z)
• Improving EEG based continuous speech recognition using GAN [3.5786621294068377]
  We demonstrate that it is possible to generate more meaningful electroencephalography (EEG) features from raw EEG features using generative adversarial networks (GAN) Our proposed approach can be implemented without using any additional sensor information, whereas in [1] authors used additional features like acoustic or articulatory information to improve the performance of EEG based continuous speech recognition systems.
  arXiv  Detail & Related papers  (2020-05-29T06:11:33Z)
• Predicting Different Acoustic Features from EEG and towards direct synthesis of Audio Waveform from EEG [3.5786621294068377]
  Authors provided preliminary results for synthesizing speech from electroencephalography (EEG) features. Deep learning model takes raw EEG waveform signals as input and directly produces audio waveform as output. Results presented in this paper shows how different acoustic features are related to non-invasive neural EEG signals recorded during speech perception and production.
  arXiv  Detail & Related papers  (2020-05-29T05:50:03Z)
• Advancing Speech Synthesis using EEG [3.5786621294068377]
  We introduce attention-regression model to demonstrate predicting acoustic features from electroencephalography (EEG) features recorded in parallel with spoken sentences. First we demonstrate predicting acoustic features directly from EEG features using our attention model and then we demonstrate predicting acoustic features from EEG features using a two-step approach.
  arXiv  Detail & Related papers  (2020-04-09T23:58:40Z)
• Speech Synthesis using EEG [4.312746668772343]
  We make use of a recurrent neural network (RNN) regression model to predict acoustic features directly from EEG features. We provide EEG based speech synthesis results for four subjects in this paper.
  arXiv  Detail & Related papers  (2020-02-22T03:53:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.