Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction

• URL: http://arxiv.org/abs/2305.02774v3
• Date: Tue, 21 May 2024 06:58:10 GMT
• Title: Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction
• Authors: Qi Wang, Zhijie Wen, Jun Shi, Qian Wang, Dinggang Shen, Shihui Ying,
• Abstract summary: We propose an end-to-end deep learning framework that utilizes T1-weighted images (T1WIs) as auxiliary modalities to expedite T2WIs' acquisitions. We employ Optimal Transport (OT) to synthesize T2WIs by aligning T1WIs and performing cross-modal synthesis. We prove that the reconstructed T2WIs and the synthetic T2WIs become closer on the T2 image manifold with iterations increasing.
• Score: 54.19448988321891
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-modal magnetic resonance imaging (MRI) plays a crucial role in comprehensive disease diagnosis in clinical medicine. However, acquiring certain modalities, such as T2-weighted images (T2WIs), is time-consuming and prone to be with motion artifacts. It negatively impacts subsequent multi-modal image analysis. To address this issue, we propose an end-to-end deep learning framework that utilizes T1-weighted images (T1WIs) as auxiliary modalities to expedite T2WIs' acquisitions. While image pre-processing is capable of mitigating misalignment, improper parameter selection leads to adverse pre-processing effects, requiring iterative experimentation and adjustment. To overcome this shortage, we employ Optimal Transport (OT) to synthesize T2WIs by aligning T1WIs and performing cross-modal synthesis, effectively mitigating spatial misalignment effects. Furthermore, we adopt an alternating iteration framework between the reconstruction task and the cross-modal synthesis task to optimize the final results. Then, we prove that the reconstructed T2WIs and the synthetic T2WIs become closer on the T2 image manifold with iterations increasing, and further illustrate that the improved reconstruction result enhances the synthesis process, whereas the enhanced synthesis result improves the reconstruction process. Finally, experimental results from FastMRI and internal datasets confirm the effectiveness of our method, demonstrating significant improvements in image reconstruction quality even at low sampling rates.

Related papers

• Two-Stage Approach for Brain MR Image Synthesis: 2D Image Synthesis and 3D Refinement [1.5683566370372715]
  It is crucial to synthesize the missing MR images that reflect the unique characteristics of the absent modality with precise tumor representation. We propose a two-stage approach that first synthesizes MR images from 2D slices using a novel intensity encoding method and then refines the synthesized MRI.
  arXiv  Detail & Related papers  (2024-10-14T08:21:08Z)
• Rotational Augmented Noise2Inverse for Low-dose Computed Tomography Reconstruction [83.73429628413773]
  Supervised deep learning methods have shown the ability to remove noise in images but require accurate ground truth. We propose a novel self-supervised framework for LDCT, in which ground truth is not required for training the convolutional neural network (CNN) Numerical and experimental results show that the reconstruction accuracy of N2I with sparse views is degrading while the proposed rotational augmented Noise2Inverse (RAN2I) method keeps better image quality over a different range of sampling angles.
  arXiv  Detail & Related papers  (2023-12-19T22:40:51Z)
• Deep Cardiac MRI Reconstruction with ADMM [7.694990352622926]
  We present a deep learning (DL)-based method for accelerated cine and multi-contrast reconstruction in the context of cardiac imaging. Our method optimize in both the image and k-space domains, allowing for high reconstruction fidelity.
  arXiv  Detail & Related papers  (2023-10-10T13:46:11Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)
• Cross-Modality High-Frequency Transformer for MR Image Super-Resolution [100.50972513285598]
  We build an early effort to build a Transformer-based MR image super-resolution framework. We consider two-fold domain priors including the high-frequency structure prior and the inter-modality context prior. We establish a novel Transformer architecture, called Cross-modality high-frequency Transformer (Cohf-T), to introduce such priors into super-resolving the low-resolution images.
  arXiv  Detail & Related papers  (2022-03-29T07:56:55Z)
• Fast T2w/FLAIR MRI Acquisition by Optimal Sampling of Information Complementary to Pre-acquired T1w MRI [52.656075914042155]
  We propose an iterative framework to optimize the under-sampling pattern for MRI acquisition of another modality. We have demonstrated superior performance of our learned under-sampling patterns on a public dataset.
  arXiv  Detail & Related papers  (2021-11-11T04:04:48Z)
• High-Resolution Pelvic MRI Reconstruction Using a Generative Adversarial Network with Attention and Cyclic Loss [3.4358954898228604]
  Super-resolution methods have shown excellent performance in accelerating MRI. In some circumstances, it is difficult to obtain high-resolution images even with prolonged scan time. We proposed a novel super-resolution method that uses a generative adversarial network (GAN) with cyclic loss and attention mechanism.
  arXiv  Detail & Related papers  (2021-07-21T10:07:22Z)
• Adaptive Gradient Balancing for UndersampledMRI Reconstruction and Image-to-Image Translation [60.663499381212425]
  We enhance the image quality by using a Wasserstein Generative Adversarial Network combined with a novel Adaptive Gradient Balancing technique. In MRI, our method minimizes artifacts, while maintaining a high-quality reconstruction that produces sharper images than other techniques.
  arXiv  Detail & Related papers  (2021-04-05T13:05:22Z)
• Dual-cycle Constrained Bijective VAE-GAN For Tagged-to-Cine Magnetic Resonance Image Synthesis [11.697141493937021]
  We propose a novel VAE-GAN approach to carry out tagged-to-cine MR image synthesis. Our framework has been trained, validated, and tested using 1,768, 416, and 1,560 subject-independent paired slices of tagged and cine MRI.
  arXiv  Detail & Related papers  (2021-01-14T03:27:16Z)
• Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis [30.64847799586407]
  We propose a multi-modality generative adversarial network (MGAN) to synthesize three high-quality MR modalities (FLAIR, T1 and T1ce) from one MR modality T2 simultaneously. The experimental results show that the quality of the synthesized images is better than the one synthesized by the baseline model, pix2pix.
  arXiv  Detail & Related papers  (2020-05-02T21:33:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.