Restore-RWKV: Efficient and Effective Medical Image Restoration with RWKV

• URL: http://arxiv.org/abs/2407.11087v3
• Date: Mon, 06 Jan 2025 15:27:33 GMT
• Title: Restore-RWKV: Efficient and Effective Medical Image Restoration with RWKV
• Authors: Zhiwen Yang, Jiayin Li, Hui Zhang, Dan Zhao, Bingzheng Wei, Yan Xu,
• Abstract summary: We propose Restore-RWKV, the first RWKV-based model for medical image restoration.<n>We present a recurrent WKV (Re-WKV) attention mechanism that captures global dependencies with linear computational complexity.<n>Experiments demonstrate that the resulting Restore-RWKV achieves SOTA performance across a range of medical image restoration tasks.
• Score: 15.585071228529731
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Transformers have revolutionized medical image restoration, but the quadratic complexity still poses limitations for their application to high-resolution medical images. The recent advent of the Receptance Weighted Key Value (RWKV) model in the natural language processing field has attracted much attention due to its ability to process long sequences efficiently. To leverage its advanced design, we propose Restore-RWKV, the first RWKV-based model for medical image restoration. Since the original RWKV model is designed for 1D sequences, we make two necessary modifications for modeling spatial relations in 2D medical images. First, we present a recurrent WKV (Re-WKV) attention mechanism that captures global dependencies with linear computational complexity. Re-WKV incorporates bidirectional attention as basic for a global receptive field and recurrent attention to effectively model 2D dependencies from various scan directions. Second, we develop an omnidirectional token shift (Omni-Shift) layer that enhances local dependencies by shifting tokens from all directions and across a wide context range. These adaptations make the proposed Restore-RWKV an efficient and effective model for medical image restoration. Even a lightweight variant of Restore-RWKV, with only 1.16 million parameters, achieves comparable or even superior results compared to existing state-of-the-art (SOTA) methods. Extensive experiments demonstrate that the resulting Restore-RWKV achieves SOTA performance across a range of medical image restoration tasks, including PET image synthesis, CT image denoising, MRI image super-resolution, and all-in-one medical image restoration. Code is available at: https://github.com/Yaziwel/Restore-RWKV.

Related papers

• UPCMR: A Universal Prompt-guided Model for Random Sampling Cardiac MRI Reconstruction [1.2773749417703923]
  We introduce UPCMR, a universal unrolled model designed for cardiac magnetic resonance imaging reconstruction. It incorporates two kinds of learnable prompts, undersampling-specific prompt and spatial-specific prompt, and integrates them with a UNet structure in each block. It highly enhances reconstructed image quality across all random sampling scenarios through an effective training strategy.
  arXiv  Detail & Related papers  (2025-02-18T07:44:35Z)
• RWKV-UNet: Improving UNet with Long-Range Cooperation for Effective Medical Image Segmentation [39.11918061481855]
  We propose RWKV-UNet, a novel model that integrates the RWKV structure into the U-Net architecture. This integration enhances the model's ability to capture long-range dependencies and improve contextual understanding. We show that RWKV-UNet achieves state-of-the-art performance on various types of medical image segmentation.
  arXiv  Detail & Related papers  (2025-01-14T22:03:00Z)
• Exploring Real&Synthetic Dataset and Linear Attention in Image Restoration [47.26304397935705]
  Image restoration aims to recover high-quality images from degraded inputs. Existing methods lack a unified training benchmark for iterations and configurations. We introduce a large-scale IR dataset called ReSyn, which employs a novel image filtering method based on image complexity.
  arXiv  Detail & Related papers  (2024-12-05T02:11:51Z)
• Empowering Image Recovery_ A Multi-Attention Approach [96.25892659985342]
  Diverse Restormer (DART) is an image restoration method that integrates information from various sources to address restoration challenges. DART employs customized attention mechanisms to enhance overall performance. evaluation across five restoration tasks consistently positions DART at the forefront.
  arXiv  Detail & Related papers  (2024-04-06T12:50:08Z)
• Deep Cardiac MRI Reconstruction with ADMM [7.694990352622926]
  We present a deep learning (DL)-based method for accelerated cine and multi-contrast reconstruction in the context of cardiac imaging. Our method optimize in both the image and k-space domains, allowing for high reconstruction fidelity.
  arXiv  Detail & Related papers  (2023-10-10T13:46:11Z)
• Fill the K-Space and Refine the Image: Prompting for Dynamic and Multi-Contrast MRI Reconstruction [31.404228406642194]
  The key to dynamic or multi-contrast magnetic resonance imaging (MRI) reconstruction lies in exploring inter-frame or inter-contrast information. We propose a two-stage MRI reconstruction pipeline to address these limitations. Our proposed method significantly outperforms previous state-of-the-art accelerated MRI reconstruction methods.
  arXiv  Detail & Related papers  (2023-09-25T02:51:00Z)
• Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction [54.19448988321891]
  We propose an end-to-end deep learning framework that utilizes T1-weighted images (T1WIs) as auxiliary modalities to expedite T2WIs' acquisitions. We employ Optimal Transport (OT) to synthesize T2WIs by aligning T1WIs and performing cross-modal synthesis. We prove that the reconstructed T2WIs and the synthetic T2WIs become closer on the T2 image manifold with iterations increasing.
  arXiv  Detail & Related papers  (2023-05-04T12:20:51Z)
• Transformer and GAN Based Super-Resolution Reconstruction Network for Medical Images [0.0]
  Super-resolution reconstruction in medical imaging has become more popular (MRI) In this paper, we offer a deep learning-based strategy for reconstructing medical images from low resolutions utilizing Transformer and Generative Adversarial Networks (T-GAN) The integrated system can extract more precise texture information and focus more on important locations through global image matching.
  arXiv  Detail & Related papers  (2022-12-26T09:52:12Z)
• R2C-GAN: Restore-to-Classify GANs for Blind X-Ray Restoration and COVID-19 Classification [17.470240973478585]
  We propose a joint model for generic image restoration and classification: Restore-to-Classify Generative Adversarial Networks (R2C-GANs) Such a jointly optimized model keeps any disease intact after the restoration. The proposed restoration approach achieves over 90% F1-Score which is significantly higher than the performance of any deep model.
  arXiv  Detail & Related papers  (2022-09-29T13:28:34Z)
• Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction [68.80715727288514]
  We show how to unfold an iterative MGDUN algorithm into a novel model-guided deep unfolding network by taking the MRI observation matrix. In this paper, we propose a novel Model-Guided interpretable Deep Unfolding Network (MGDUN) for medical image SR reconstruction.
  arXiv  Detail & Related papers  (2022-09-15T03:58:30Z)
• Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution [55.52779466954026]
  Multi-contrast super-resolution (SR) reconstruction is promising to yield SR images with higher quality. Existing methods lack effective mechanisms to match and fuse these features for better reconstruction. We propose a novel network to address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques.
  arXiv  Detail & Related papers  (2022-03-26T01:42:59Z)
• ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer [60.27951773998535]
  We propose a recurrent transformer model, namely textbfReconFormer, for MRI reconstruction. It can iteratively reconstruct high fertility magnetic resonance images from highly under-sampled k-space data. We show that it achieves significant improvements over the state-of-the-art methods with better parameter efficiency.
  arXiv  Detail & Related papers  (2022-01-23T21:58:19Z)
• Multi-Modal MRI Reconstruction with Spatial Alignment Network [51.74078260367654]
  In clinical practice, magnetic resonance imaging (MRI) with multiple contrasts is usually acquired in a single study. Recent researches demonstrate that, considering the redundancy between different contrasts or modalities, a target MRI modality under-sampled in the k-space can be better reconstructed with the helps from a fully-sampled sequence. In this paper, we integrate the spatial alignment network with reconstruction, to improve the quality of the reconstructed target modality.
  arXiv  Detail & Related papers  (2021-08-12T08:46:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.