A Unified Framework for Synthesizing Multisequence Brain MRI via Hybrid Fusion

• URL: http://arxiv.org/abs/2406.14954v1
• Date: Fri, 21 Jun 2024 08:06:00 GMT
• Title: A Unified Framework for Synthesizing Multisequence Brain MRI via Hybrid Fusion
• Authors: Jihoon Cho, Jonghye Woo, Jinah Park,
• Abstract summary: We propose a novel unified framework for synthesizing multisequence MR images, called Hybrid Fusion GAN (HF-GAN) We introduce a hybrid fusion encoder designed to ensure the disentangled extraction of complementary and modality-specific information. Common feature representations are transformed into a target latent space via the modality infuser to synthesize missing MR sequences.
• Score: 4.47838172826189
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multisequence Magnetic Resonance Imaging (MRI) provides a reliable diagnosis in clinical applications through complementary information within sequences. However, in practice, the absence of certain MR sequences is a common problem that can lead to inconsistent analysis results. In this work, we propose a novel unified framework for synthesizing multisequence MR images, called Hybrid Fusion GAN (HF-GAN). We introduce a hybrid fusion encoder designed to ensure the disentangled extraction of complementary and modality-specific information, along with a channel attention-based feature fusion module that integrates the features into a common latent space handling the complexity from combinations of accessible MR sequences. Common feature representations are transformed into a target latent space via the modality infuser to synthesize missing MR sequences. We have performed experiments on multisequence brain MRI datasets from healthy individuals and patients diagnosed with brain tumors. Experimental results show that our method outperforms state-of-the-art methods in both quantitative and qualitative comparisons. In addition, a detailed analysis of our framework demonstrates the superiority of our designed modules and their effectiveness for use in data imputation tasks.

Related papers

• Towards General Text-guided Image Synthesis for Customized Multimodal Brain MRI Generation [51.28453192441364]
  Multimodal brain magnetic resonance (MR) imaging is indispensable in neuroscience and neurology. Current MR image synthesis approaches are typically trained on independent datasets for specific tasks. We present TUMSyn, a Text-guided Universal MR image Synthesis model, which can flexibly generate brain MR images.
  arXiv  Detail & Related papers  (2024-09-25T11:14:47Z)
• Disentangled Multimodal Brain MR Image Translation via Transformer-based Modality Infuser [12.402947207350394]
  We propose a transformer-based modality infuser designed to synthesize multimodal brain MR images. In our method, we extract modality-agnostic features from the encoder and then transform them into modality-specific features. We carried out experiments on the BraTS 2018 dataset, translating between four MR modalities.
  arXiv  Detail & Related papers  (2024-02-01T06:34:35Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• BrainVoxGen: Deep learning framework for synthesis of Ultrasound to MRI [2.982610402087728]
  The work proposes a novel deep-learning framework for the synthesis of three-dimensional MRI volumes from corresponding 3D ultrasound images of the brain. This research holds promise for transformative applications in medical diagnostics and treatment planning within the neuroimaging domain.
  arXiv  Detail & Related papers  (2023-10-11T20:37:59Z)
• K-Space-Aware Cross-Modality Score for Synthesized Neuroimage Quality Assessment [71.27193056354741]
  The problem of how to assess cross-modality medical image synthesis has been largely unexplored. We propose a new metric K-CROSS to spur progress on this challenging problem. K-CROSS uses a pre-trained multi-modality segmentation network to predict the lesion location.
  arXiv  Detail & Related papers  (2023-07-10T01:26:48Z)
• IMPORTANT-Net: Integrated MRI Multi-Parameter Reinforcement Fusion Generator with Attention Network for Synthesizing Absent Data [16.725225424047256]
  We develop a novel $textbfI$ntegrated MRI $textbfM$ulti-$textbfP$arameter reinf$textbfO$rcement fusion generato$textbfR$ wi$textbfT$h. We show that our IMPORTANT-Net is capable of generating missing MRI parameters and outperforms comparable state-of-the-art networks.
  arXiv  Detail & Related papers  (2023-02-03T14:56:10Z)
• A Novel Unified Conditional Score-based Generative Framework for Multi-modal Medical Image Completion [54.512440195060584]
  We propose the Unified Multi-Modal Conditional Score-based Generative Model (UMM-CSGM) to take advantage of Score-based Generative Model (SGM) UMM-CSGM employs a novel multi-in multi-out Conditional Score Network (mm-CSN) to learn a comprehensive set of cross-modal conditional distributions. Experiments on BraTS19 dataset show that the UMM-CSGM can more reliably synthesize the heterogeneous enhancement and irregular area in tumor-induced lesions.
  arXiv  Detail & Related papers  (2022-07-07T16:57:21Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Deep Learning based Multi-modal Computing with Feature Disentanglement for MRI Image Synthesis [8.363448006582065]
  We propose a deep learning based multi-modal computing model for MRI synthesis with feature disentanglement strategy. The proposed approach decomposes each input modality into modality-invariant space with shared information and modality-specific space with specific information. To address the lack of specific information of the target modality in the test phase, a local adaptive fusion (LAF) module is adopted to generate a modality-like pseudo-target.
  arXiv  Detail & Related papers  (2021-05-06T17:22:22Z)
• Mapping individual differences in cortical architecture using multi-view representation learning [0.0]
  We introduce a novel machine learning method which allows combining the activation-and connectivity-based information respectively measured through task-fMRI and resting-state fMRI. It combines a multi-view deep autoencoder which is designed to fuse the two fMRI modalities into a joint representation space within which a predictive model is trained to guess a scalar score that characterizes the patient.
  arXiv  Detail & Related papers  (2020-04-01T09:01:25Z)
• Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis [143.55901940771568]
  We propose a novel Hybrid-fusion Network (Hi-Net) for multi-modal MR image synthesis. In our Hi-Net, a modality-specific network is utilized to learn representations for each individual modality. A multi-modal synthesis network is designed to densely combine the latent representation with hierarchical features from each modality.
  arXiv  Detail & Related papers  (2020-02-11T08:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.