Reconstructing Retinal Visual Images from 3T fMRI Data Enhanced by Unsupervised Learning

• URL: http://arxiv.org/abs/2404.05107v1
• Date: Sun, 7 Apr 2024 23:31:37 GMT
• Title: Reconstructing Retinal Visual Images from 3T fMRI Data Enhanced by Unsupervised Learning
• Authors: Yujian Xiong, Wenhui Zhu, Zhong-Lin Lu, Yalin Wang,
• Abstract summary: We propose a novel framework that generates enhanced 3T fMRI data through an unsupervised Generative Adversarial Network (GAN) In this paper, we demonstrate the reconstruction capabilities of the enhanced 3T fMRI data, highlighting its proficiency in generating superior input visual images.
• Score: 2.1597860906272803
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The reconstruction of human visual inputs from brain activity, particularly through functional Magnetic Resonance Imaging (fMRI), holds promising avenues for unraveling the mechanisms of the human visual system. Despite the significant strides made by deep learning methods in improving the quality and interpretability of visual reconstruction, there remains a substantial demand for high-quality, long-duration, subject-specific 7-Tesla fMRI experiments. The challenge arises in integrating diverse smaller 3-Tesla datasets or accommodating new subjects with brief and low-quality fMRI scans. In response to these constraints, we propose a novel framework that generates enhanced 3T fMRI data through an unsupervised Generative Adversarial Network (GAN), leveraging unpaired training across two distinct fMRI datasets in 7T and 3T, respectively. This approach aims to overcome the limitations of the scarcity of high-quality 7-Tesla data and the challenges associated with brief and low-quality scans in 3-Tesla experiments. In this paper, we demonstrate the reconstruction capabilities of the enhanced 3T fMRI data, highlighting its proficiency in generating superior input visual images compared to data-intensive methods trained and tested on a single subject.

Related papers

• MRI Reconstruction with Regularized 3D Diffusion Model (R3DM) [2.842800539489865]
  We propose a 3D MRI reconstruction method that leverages a regularized 3D diffusion model combined with optimization method. By incorporating diffusion based priors, our method improves image quality, reduces noise, and enhances the overall fidelity of 3D MRI reconstructions.
  arXiv  Detail & Related papers  (2024-12-25T00:55:05Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• 7T MRI Synthesization from 3T Acquisitions [1.1549572298362787]
  Supervised deep learning techniques can be used to generate synthetic 7T MRIs from 3T MRI inputs. In this paper, we introduce multiple novel 7T synthesization algorithms based on custom-designed variants of the V-Net convolutional neural network.
  arXiv  Detail & Related papers  (2024-03-13T22:06:44Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• Transferring Ultrahigh-Field Representations for Intensity-Guided Brain Segmentation of Low-Field Magnetic Resonance Imaging [51.92395928517429]
  The use of 7T MRI is limited by its high cost and lower accessibility compared to low-field (LF) MRI. This study proposes a deep-learning framework that fuses the input LF magnetic resonance feature representations with the inferred 7T-like feature representations for brain image segmentation tasks.
  arXiv  Detail & Related papers  (2024-02-13T12:21:06Z)
• MinD-3D: Reconstruct High-quality 3D objects in Human Brain [50.534007259536715]
  Recon3DMind is an innovative task aimed at reconstructing 3D visuals from Functional Magnetic Resonance Imaging (fMRI) signals. We present the fMRI-Shape dataset, which includes data from 14 participants and features 360-degree videos of 3D objects. We propose MinD-3D, a novel and effective three-stage framework specifically designed to decode the brain's 3D visual information from fMRI signals.
  arXiv  Detail & Related papers  (2023-12-12T18:21:36Z)
• fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding [54.17776744076334]
  We propose fMRI-PTE, an innovative auto-encoder approach for fMRI pre-training. Our approach involves transforming fMRI signals into unified 2D representations, ensuring consistency in dimensions and preserving brain activity patterns. Our contributions encompass introducing fMRI-PTE, innovative data transformation, efficient training, a novel learning strategy, and the universal applicability of our approach.
  arXiv  Detail & Related papers  (2023-11-01T07:24:22Z)
• Leveraging Neural Radiance Fields for Uncertainty-Aware Visual Localization [56.95046107046027]
  We propose to leverage Neural Radiance Fields (NeRF) to generate training samples for scene coordinate regression. Despite NeRF's efficiency in rendering, many of the rendered data are polluted by artifacts or only contain minimal information gain.
  arXiv  Detail & Related papers  (2023-10-10T20:11:13Z)
• Two-and-a-half Order Score-based Model for Solving 3D Ill-posed Inverse Problems [7.074380879971194]
  We propose a novel two-and-a-half order score-based model (TOSM) for 3D volumetric reconstruction. During the training phase, our TOSM learns data distributions in 2D space, which reduces the complexity of training. In the reconstruction phase, the TOSM updates the data distribution in 3D space, utilizing complementary scores along three directions.
  arXiv  Detail & Related papers  (2023-08-16T17:07:40Z)
• Iterative Data Refinement for Self-Supervised MR Image Reconstruction [18.02961646651716]
  We propose a data refinement framework for self-supervised MR image reconstruction. We first analyze the reason of the performance gap between self-supervised and supervised methods. Then, we design an effective self-supervised training data refinement method to reduce this data bias.
  arXiv  Detail & Related papers  (2022-11-24T06:57:16Z)
• Synthetic Magnetic Resonance Images with Generative Adversarial Networks [0.0]
  In this work, we experiment three GAN architectures with different loss functions to generate new brain MRIs. The results show the importance of hyper parameter tuning and the use of mini-batch similarity layer in the Discriminator and gradient penalty in the loss function to achieve convergence with high quality and realism.
  arXiv  Detail & Related papers  (2020-01-17T11:00:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.