From Brainwaves to Brain Scans: A Robust Neural Network for EEG-to-fMRI Synthesis

• URL: http://arxiv.org/abs/2502.08025v3
• Date: Mon, 28 Apr 2025 16:57:29 GMT
• Title: From Brainwaves to Brain Scans: A Robust Neural Network for EEG-to-fMRI Synthesis
• Authors: Kristofer Grover Roos, Atsushi Fukuda, Quan Huu Cap,
• Abstract summary: We propose E2fNet, a simple yet effective deep learning model for synthesizing fMRI images from low-cost EEG data.<n>E2fNet is an encoder-decoder network specifically designed to capture and translate meaningful multi-scale features from EEG across electrode channels into accurate fMRI representations.
• Score: 4.710921988115686
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While functional magnetic resonance imaging (fMRI) offers valuable insights into brain activity, it is limited by high operational costs and significant infrastructural demands. In contrast, electroencephalography (EEG) provides millisecond-level precision in capturing electrical activity but lacks the spatial fidelity necessary for precise neural localization. To bridge these gaps, we propose E2fNet, a simple yet effective deep learning model for synthesizing fMRI images from low-cost EEG data. E2fNet is an encoder-decoder network specifically designed to capture and translate meaningful multi-scale features from EEG across electrode channels into accurate fMRI representations. Extensive evaluations across three public datasets demonstrate that E2fNet consistently outperforms existing CNN- and transformer-based methods, achieving state-of-the-art results in terms of the structural similarity index measure (SSIM). These results demonstrate that E2fNet is a promising, cost-effective solution for enhancing neuroimaging capabilities. The code is available at https://github.com/kgr20/E2fNet.

Related papers

• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals.<n>Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality.<n>The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• EEG-DCNet: A Fast and Accurate MI-EEG Dilated CNN Classification Method [10.791605945979995]
  We present a novel multi-scale atrous convolutional neural network (CNN) model called EEG-dilated convolution network (DCNet) We incorporate the $1times1$ convolutional layer and utilize the multi-branch parallel atrous convolutional architecture in EEG-DCNet. We show that EEG-DCNet outperforms existing state-of-the-art (SOTA) approaches in terms of classification accuracy and Kappa scores.
  arXiv  Detail & Related papers  (2024-11-12T09:47:50Z)
• Neuromorphic Wireless Split Computing with Multi-Level Spikes [69.73249913506042]
  Neuromorphic computing uses spiking neural networks (SNNs) to perform inference tasks. embedding a small payload within each spike exchanged between spiking neurons can enhance inference accuracy without increasing energy consumption. split computing - where an SNN is partitioned across two devices - is a promising solution. This paper presents the first comprehensive study of a neuromorphic wireless split computing architecture that employs multi-level SNNs.
  arXiv  Detail & Related papers  (2024-11-07T14:08:35Z)
• A Unified Model for Compressed Sensing MRI Across Undersampling Patterns [69.19631302047569]
  Deep neural networks have shown great potential for reconstructing high-fidelity images from undersampled measurements. Our model is based on neural operators, a discretization-agnostic architecture. Our inference speed is also 1,400x faster than diffusion methods.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• NT-ViT: Neural Transcoding Vision Transformers for EEG-to-fMRI Synthesis [7.542742087154667]
  This paper introduces the Neural Transcoding Vision Transformer (modelname) modelname is a generative model designed to estimate high-resolution functional Magnetic Resonance Imaging (fMRI) samples from simultaneous Electroencephalography (EEG) data.
  arXiv  Detail & Related papers  (2024-09-18T09:38:08Z)
• Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models [4.933734706786783]
  EEG is a low-cost, non-invasive, and portable neuroimaging technique.<n>EEG presents inherent challenges due to its low spatial resolution and susceptibility to noise and artifacts.<n>We propose a framework based on the ControlNet adapter for conditioning a latent diffusion model through EEG signals.
  arXiv  Detail & Related papers  (2024-09-17T19:07:13Z)
• Prototype Learning Guided Hybrid Network for Breast Tumor Segmentation in DCE-MRI [58.809276442508256]
  We propose a hybrid network via the combination of convolution neural network (CNN) and transformer layers. The experimental results on private and public DCE-MRI datasets demonstrate that the proposed hybrid network superior performance than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-11T15:46:00Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Leveraging sinusoidal representation networks to predict fMRI signals from EEG [3.3121941932506473]
  We propose a novel architecture that can predict fMRI signals directly from multi-channel EEG without explicit feature engineering. Our model achieves this by implementing a Sinusoidal Representation Network (SIREN) to learn frequency information in brain dynamics. We evaluate our model using a simultaneous EEG-fMRI dataset with 8 subjects and investigate its potential for predicting subcortical fMRI signals.
  arXiv  Detail & Related papers  (2023-11-06T03:16:18Z)
• Joint fMRI Decoding and Encoding with Latent Embedding Alignment [77.66508125297754]
  We introduce a unified framework that addresses both fMRI decoding and encoding. Our model concurrently recovers visual stimuli from fMRI signals and predicts brain activity from images within a unified framework.
  arXiv  Detail & Related papers  (2023-03-26T14:14:58Z)
• Functional Magnetic Resonance Imaging data augmentation through conditional ICA [44.483210864902304]
  We introduce Conditional Independent Components Analysis (Conditional ICA): a fast functional Magnetic Resonance Imaging (fMRI) data augmentation technique. We show that Conditional ICA is successful at synthesizing data indistinguishable from observations, and that it yields gains in classification accuracy in brain decoding problems.
  arXiv  Detail & Related papers  (2021-07-11T22:36:14Z)
• Learning Frequency-aware Dynamic Network for Efficient Super-Resolution [56.98668484450857]
  This paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures.
  arXiv  Detail & Related papers  (2021-03-15T12:54:26Z)
• EEG-Inception: An Accurate and Robust End-to-End Neural Network for EEG-based Motor Imagery Classification [123.93460670568554]
  This paper proposes a novel convolutional neural network (CNN) architecture for accurate and robust EEG-based motor imagery (MI) classification. The proposed CNN model, namely EEG-Inception, is built on the backbone of the Inception-Time network. The proposed network is an end-to-end classification, as it takes the raw EEG signals as the input and does not require complex EEG signal-preprocessing.
  arXiv  Detail & Related papers  (2021-01-24T19:03:10Z)
• EEG to fMRI Synthesis: Is Deep Learning a candidate? [0.913755431537592]
  This work provides the first comprehensive on how to use state-of-the-art principles from Neural Processing to synthesize fMRI data from electroencephalographic (EEG) view data. A comparison of state-of-the-art synthesis approaches, including Autoencoders, Generative Adrial Networks and Pairwise Learning, is undertaken. Results highlight the feasibility of EEG to fMRI brain image mappings, pinpointing the role of current advances in Machine Learning and showing the relevance of upcoming contributions to further improve performance.
  arXiv  Detail & Related papers  (2020-09-29T16:29:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.