Magnetic Resonance Image Processing Transformer for General Accelerated Image Reconstruction

• URL: http://arxiv.org/abs/2405.15098v2
• Date: Fri, 07 Feb 2025 21:02:23 GMT
• Title: Magnetic Resonance Image Processing Transformer for General Accelerated Image Reconstruction
• Authors: Guoyao Shen, Mengyu Li, Stephan Anderson, Chad W. Farris, Xin Zhang,
• Abstract summary: We introduce the Magnetic Resonance Image Processing Transformer (MR-IPT), a ViT-based framework designed to enhance the generalizability and robustness of accelerated MRI reconstruction. By leveraging a shared transformer backbone, MR-IPT effectively learns universal feature representations, allowing it to generalize across diverse reconstruction tasks. Our findings suggest that transformer-based general models can significantly advance MRI reconstruction, offering improved adaptability and stability compared to traditional deep learning approaches.
• Score: 2.7802624923496353
• License:
• Abstract: Recent advancements in deep learning have enabled the development of generalizable models that achieve state-of-the-art performance across various imaging tasks. Vision Transformer (ViT)-based architectures, in particular, have demonstrated strong feature extraction capabilities when pre-trained on large-scale datasets. In this work, we introduce the Magnetic Resonance Image Processing Transformer (MR-IPT), a ViT-based framework designed to enhance the generalizability and robustness of accelerated MRI reconstruction. Unlike conventional deep learning models that require separate training for different acceleration factors, MR-IPT is pre-trained on a large-scale dataset encompassing multiple undersampling patterns and acceleration settings, enabling a unified reconstruction framework. By leveraging a shared transformer backbone, MR-IPT effectively learns universal feature representations, allowing it to generalize across diverse reconstruction tasks. Extensive experiments demonstrate that MR-IPT outperforms both CNN-based and existing transformer-based methods, achieving superior reconstruction quality across varying acceleration factors and sampling masks. Moreover, MR-IPT exhibits strong robustness, maintaining high performance even under unseen acquisition setups, highlighting its potential as a scalable and efficient solution for accelerated MRI. Our findings suggest that transformer-based general models can significantly advance MRI reconstruction, offering improved adaptability and stability compared to traditional deep learning approaches.

Related papers

• STMR: Spiral Transformer for Hand Mesh Reconstruction [11.47314655711821]
  This paper ingeniously integrates spiral sampling into the Transformer architecture, enhancing its ability to leverage mesh topology for superior performance in hand mesh reconstruction. Experiments on the FreiHAND dataset demonstrate the state-of-the-art performance and unparalleled inference speed of STMR compared with similar backbone methods.
  arXiv  Detail & Related papers  (2024-07-08T14:05:27Z)
• Cross-Modality High-Frequency Transformer for MR Image Super-Resolution [100.50972513285598]
  We build an early effort to build a Transformer-based MR image super-resolution framework. We consider two-fold domain priors including the high-frequency structure prior and the inter-modality context prior. We establish a novel Transformer architecture, called Cross-modality high-frequency Transformer (Cohf-T), to introduce such priors into super-resolving the low-resolution images.
  arXiv  Detail & Related papers  (2022-03-29T07:56:55Z)
• Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution [55.52779466954026]
  Multi-contrast super-resolution (SR) reconstruction is promising to yield SR images with higher quality. Existing methods lack effective mechanisms to match and fuse these features for better reconstruction. We propose a novel network to address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques.
  arXiv  Detail & Related papers  (2022-03-26T01:42:59Z)
• Fast MRI Reconstruction: How Powerful Transformers Are? [1.523157765626545]
  Methods by k-space undersampling and deep learning based reconstruction have been popularised to accelerate the scanning process. In particular, a generative adversarial network (GAN) based Swin transformer (ST-GAN) was introduced for the fast MRI reconstruction. We show that transformers work well for the MRI reconstruction from different undersampling conditions.
  arXiv  Detail & Related papers  (2022-01-23T23:41:48Z)
• ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer [60.27951773998535]
  We propose a recurrent transformer model, namely textbfReconFormer, for MRI reconstruction. It can iteratively reconstruct high fertility magnetic resonance images from highly under-sampled k-space data. We show that it achieves significant improvements over the state-of-the-art methods with better parameter efficiency.
  arXiv  Detail & Related papers  (2022-01-23T21:58:19Z)
• Reference-based Magnetic Resonance Image Reconstruction Using Texture Transforme [86.6394254676369]
  We propose a novel Texture Transformer Module (TTM) for accelerated MRI reconstruction. We formulate the under-sampled data and reference data as queries and keys in a transformer. The proposed TTM can be stacked on prior MRI reconstruction approaches to further improve their performance.
  arXiv  Detail & Related papers  (2021-11-18T03:06:25Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Accelerated Multi-Modal MR Imaging with Transformers [92.18406564785329]
  We propose a multi-modal transformer (MTrans) for accelerated MR imaging. By restructuring the transformer architecture, our MTrans gains a powerful ability to capture deep multi-modal information. Our framework provides two appealing benefits: (i) MTrans is the first attempt at using improved transformers for multi-modal MR imaging, affording more global information compared with CNN-based methods.
  arXiv  Detail & Related papers  (2021-06-27T15:01:30Z)
• Robust Image Reconstruction with Misaligned Structural Information [0.27074235008521236]
  We propose a variational framework which jointly performs reconstruction and registration. Our approach is the first to achieve this for different modalities and outranks established approaches in terms of accuracy of both reconstruction and registration.
  arXiv  Detail & Related papers  (2020-04-01T17:21:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.