From Brainwaves to Brain Scans: A Robust Neural Network for EEG-to-fMRI Synthesis

• URL: http://arxiv.org/abs/2502.08025v2
• Date: Sat, 15 Feb 2025 07:08:32 GMT
• Title: From Brainwaves to Brain Scans: A Robust Neural Network for EEG-to-fMRI Synthesis
• Authors: Kristofer Grover Roos, Atsushi Fukuda, Quan Huu Cap,
• Abstract summary: We introduce E2fNet, a simple yet effective deep learning model for synthesizing fMRI images from low-cost EEG data. E2fNet is specifically designed to capture and translate meaningful features from EEG across electrode channels into accurate fMRI representations. Our findings suggest that E2fNet is a promising, cost-effective solution for enhancing neuroimaging capabilities.
• Score: 4.710921988115686
• License:
• Abstract: While functional magnetic resonance imaging (fMRI) offers rich spatial resolution, it is limited by high operational costs and significant infrastructural demands. In contrast, electroencephalography (EEG) provides millisecond-level precision in capturing electrical activity but lacks the spatial resolution necessary for precise neural localization. To bridge these gaps, we introduce E2fNet, a simple yet effective deep learning model for synthesizing fMRI images from low-cost EEG data. E2fNet is specifically designed to capture and translate meaningful features from EEG across electrode channels into accurate fMRI representations. Extensive evaluations across three datasets demonstrate that E2fNet consistently outperforms existing methods, achieving state-of-the-art results in terms of the structural similarity index measure (SSIM). Our findings suggest that E2fNet is a promising, cost-effective solution for enhancing neuroimaging capabilities. The code is available at https://github.com/kgr20/E2fNet.

Related papers

• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals. Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality. The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• EEG-DCNet: A Fast and Accurate MI-EEG Dilated CNN Classification Method [10.791605945979995]
  We present a novel multi-scale atrous convolutional neural network (CNN) model called EEG-dilated convolution network (DCNet) We incorporate the $1times1$ convolutional layer and utilize the multi-branch parallel atrous convolutional architecture in EEG-DCNet. We show that EEG-DCNet outperforms existing state-of-the-art (SOTA) approaches in terms of classification accuracy and Kappa scores.
  arXiv  Detail & Related papers  (2024-11-12T09:47:50Z)
• Neuromorphic Wireless Split Computing with Multi-Level Spikes [69.73249913506042]
  Neuromorphic computing uses spiking neural networks (SNNs) to perform inference tasks. embedding a small payload within each spike exchanged between spiking neurons can enhance inference accuracy without increasing energy consumption. split computing - where an SNN is partitioned across two devices - is a promising solution. This paper presents the first comprehensive study of a neuromorphic wireless split computing architecture that employs multi-level SNNs.
  arXiv  Detail & Related papers  (2024-11-07T14:08:35Z)
• NT-ViT: Neural Transcoding Vision Transformers for EEG-to-fMRI Synthesis [7.542742087154667]
  This paper introduces the Neural Transcoding Vision Transformer (modelname) modelname is a generative model designed to estimate high-resolution functional Magnetic Resonance Imaging (fMRI) samples from simultaneous Electroencephalography (EEG) data.
  arXiv  Detail & Related papers  (2024-09-18T09:38:08Z)
• Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models [4.933734706786783]
  EEG is a low-cost, non-invasive, and portable neuroimaging technique. EEG presents inherent challenges due to its low spatial resolution and susceptibility to noise and artifacts. We propose a framework based on the ControlNet adapter for conditioning a latent diffusion model through EEG signals.
  arXiv  Detail & Related papers  (2024-09-17T19:07:13Z)
• Brain3D: Generating 3D Objects from fMRI [76.41771117405973]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject. We show that our model captures the distinct functionalities of each region of human vision system. Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• Leveraging sinusoidal representation networks to predict fMRI signals from EEG [3.3121941932506473]
  We propose a novel architecture that can predict fMRI signals directly from multi-channel EEG without explicit feature engineering. Our model achieves this by implementing a Sinusoidal Representation Network (SIREN) to learn frequency information in brain dynamics. We evaluate our model using a simultaneous EEG-fMRI dataset with 8 subjects and investigate its potential for predicting subcortical fMRI signals.
  arXiv  Detail & Related papers  (2023-11-06T03:16:18Z)
• DynDepNet: Learning Time-Varying Dependency Structures from fMRI Data via Dynamic Graph Structure Learning [58.94034282469377]
  We propose DynDepNet, a novel method for learning the optimal time-varying dependency structure of fMRI data induced by downstream prediction tasks. Experiments on real-world fMRI datasets, for the task of sex classification, demonstrate that DynDepNet achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2022-09-27T16:32:11Z)
• EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification [123.93460670568554]
  This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based motor imagery (MI) classification. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network. The proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing.
  arXiv  Detail & Related papers  (2021-01-24T19:03:10Z)
• EEG to fMRI Synthesis: Is Deep Learning a candidate? [0.913755431537592]
  This work provides the first comprehensive on how to use state-of-the-art principles from Neural Processing to synthesize fMRI data from electroencephalographic (EEG) view data. A comparison of state-of-the-art synthesis approaches, including Autoencoders, Generative Adrial Networks and Pairwise Learning, is undertaken. Results highlight the feasibility of EEG to fMRI brain image mappings, pinpointing the role of current advances in Machine Learning and showing the relevance of upcoming contributions to further improve performance.
  arXiv  Detail & Related papers  (2020-09-29T16:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.