A Survey of fMRI to Image Reconstruction

• URL: http://arxiv.org/abs/2502.16861v1
• Date: Mon, 24 Feb 2025 05:53:04 GMT
• Title: A Survey of fMRI to Image Reconstruction
• Authors: Weiyu Guo, Guoying Sun, JianXiang He, Tong Shao, Shaoguang Wang, Ziyang Chen, Meisheng Hong, Ying Sun, Hui Xiong,
• Abstract summary: Functional magnetic resonance imaging (fMRI) based image reconstruction plays a pivotal role in decoding human perception.<n>Recent advancements in deep learning have driven progress, but challenges such as data scarcity, cross-subject variability, and low semantic consistency persist.<n>We introduce the concept of fMRI-to-Image Learning (fMRI2Image) and present the first systematic review in this field.
• Score: 21.906165069577895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Functional magnetic resonance imaging (fMRI) based image reconstruction plays a pivotal role in decoding human perception, with applications in neuroscience and brain-computer interfaces. While recent advancements in deep learning and large-scale datasets have driven progress, challenges such as data scarcity, cross-subject variability, and low semantic consistency persist. To address these issues, we introduce the concept of fMRI-to-Image Learning (fMRI2Image) and present the first systematic review in this field. This review highlights key challenges, categorizes methodologies such as fMRI signal encoding, feature mapping, and image generator. Finally, promising research directions are proposed to advance this emerging frontier, providing a reference for future studies.

Related papers

• A Comprehensive Survey on Magnetic Resonance Image Reconstruction [6.157032790444799]
  Deep learning-based MRI reconstruction has made significant progress in recent years. However, MRI reconstruction remains a challenging problem that has yet to be fully resolved.
  arXiv  Detail & Related papers  (2025-03-10T09:20:53Z)
• ContextMRI: Enhancing Compressed Sensing MRI through Metadata Conditioning [51.26601171361753]
  We propose ContextMRI, a text-conditioned diffusion model for MRI that integrates granular metadata into the reconstruction process.<n>We show that increasing the fidelity of metadata, ranging from slice location and contrast to patient age, sex, and pathology, systematically boosts reconstruction performance.
  arXiv  Detail & Related papers  (2025-01-08T05:15:43Z)
• MindFormer: Semantic Alignment of Multi-Subject fMRI for Brain Decoding [50.55024115943266]
  We introduce a novel semantic alignment method of multi-subject fMRI signals using so-called MindFormer. This model is specifically designed to generate fMRI-conditioned feature vectors that can be used for conditioning Stable Diffusion model for fMRI- to-image generation or large language model (LLM) for fMRI-to-text generation. Our experimental results demonstrate that MindFormer generates semantically consistent images and text across different subjects.
  arXiv  Detail & Related papers  (2024-05-28T00:36:25Z)
• Volumetric Reconstruction Resolves Off-Resonance Artifacts in Static and Dynamic PROPELLER MRI [76.60362295758596]
  Off-resonance artifacts in magnetic resonance imaging (MRI) are visual distortions that occur when the actual resonant frequencies of spins within the imaging volume differ from the expected frequencies used to encode spatial information. We propose to resolve these artifacts by lifting the 2D MRI reconstruction problem to 3D, introducing an additional "spectral" dimension to model this off-resonance.
  arXiv  Detail & Related papers  (2023-11-22T05:44:51Z)
• fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding [54.17776744076334]
  We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
  arXiv  Detail & Related papers  (2023-11-01T07:24:22Z)
• CMRxRecon: An open cardiac MRI dataset for the competition of accelerated image reconstruction [62.61209705638161]
  There has been growing interest in deep learning-based CMR imaging algorithms. Deep learning methods require large training datasets. This dataset includes multi-contrast, multi-view, multi-slice and multi-coil CMR imaging data from 300 subjects.
  arXiv  Detail & Related papers  (2023-09-19T15:14:42Z)
• Deep Learning for Retrospective Motion Correction in MRI: A Comprehensive Review [10.968260853546797]
  Motion represents one of the major challenges in magnetic resonance imaging (MRI) Deep learning has been frequently proposed for motion correction at several stages of the reconstruction process. This review identifies differences and synergies in underlying data usage, architectures, training and evaluation strategies.
  arXiv  Detail & Related papers  (2023-05-11T11:50:32Z)
• Facial Image Reconstruction from Functional Magnetic Resonance Imaging via GAN Inversion with Improved Attribute Consistency [5.705640492618758]
  We propose a new framework to reconstruct facial images from fMRI data. The proposed framework accomplishes two goals: (1) reconstructing clear facial images from fMRI data and (2) maintaining the consistency of semantic characteristics.
  arXiv  Detail & Related papers  (2022-07-03T11:18:35Z)
• Data and Physics Driven Learning Models for Fast MRI -- Fundamentals and Methodologies from CNN, GAN to Attention and Transformers [72.047680167969]
  This article aims to introduce the deep learning based data driven techniques for fast MRI including convolutional neural network and generative adversarial network based methods. We will detail the research in coupling physics and data driven models for MRI acceleration. Finally, we will demonstrate through a few clinical applications, explain the importance of data harmonisation and explainable models for such fast MRI techniques in multicentre and multi-scanner studies.
  arXiv  Detail & Related papers  (2022-04-01T22:48:08Z)
• Multi-Coil MRI Reconstruction Challenge -- Assessing Brain MRI Reconstruction Models and their Generalizability to Varying Coil Configurations [40.263770807921524]
  Deep-learning-based brain magnetic resonance imaging (MRI) reconstruction methods have the potential to accelerate the MRI acquisition process. The Multi-Coil Magnetic Resonance Image (MC-MRI) Reconstruction Challenge provides a benchmark that aims at addressing these issues. We describe the challenge experimental design, and summarize the results of a set of baseline and state of the art brain MRI reconstruction models.
  arXiv  Detail & Related papers  (2020-11-10T04:11:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.