Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution

• URL: http://arxiv.org/abs/2204.04218v2
• Date: Tue, 12 Apr 2022 06:18:34 GMT
• Title: Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution
• Authors: Mariana-Iuliana Georgescu, Radu Tudor Ionescu, Andreea-Iuliana Miron, Olivian Savencu, Nicolae-Catalin Ristea, Nicolae Verga, Fahad Shahbaz Khan
• Abstract summary: We propose a novel multi-head convolutional attention module to super-resolve CT and MRI scans. Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple input tensors. We introduce multiple attention heads, each head having a distinct receptive field size corresponding to a particular reduction rate for the spatial attention.
• Score: 56.622832383316215
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Super-resolving medical images can help physicians in providing more accurate diagnostics. In many situations, computed tomography (CT) or magnetic resonance imaging (MRI) techniques output several scans (modes) during a single investigation, which can jointly be used (in a multimodal fashion) to further boost the quality of super-resolution results. To this end, we propose a novel multimodal multi-head convolutional attention module to super-resolve CT and MRI scans. Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple concatenated input tensors, where the kernel (receptive field) size controls the reduction rate of the spatial attention and the number of convolutional filters controls the reduction rate of the channel attention, respectively. We introduce multiple attention heads, each head having a distinct receptive field size corresponding to a particular reduction rate for the spatial attention. We integrate our multimodal multi-head convolutional attention (MMHCA) into two deep neural architectures for super-resolution and conduct experiments on three data sets. Our empirical results show the superiority of our attention module over the state-of-the-art attention mechanisms used in super-resolution. Moreover, we conduct an ablation study to assess the impact of the components involved in our attention module, e.g. the number of inputs or the number of heads.

Related papers

• Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention [1.1155836879100416]
  We propose a Modality-agnostic Domain Generalizable Network (MADGNet) for medical image segmentation. MFMSA block refines the process of spatial feature extraction, particularly in capturing boundary features. E-SDM mitigates information loss in multi-task learning with deep supervision.
  arXiv  Detail & Related papers  (2024-05-10T07:34:36Z)
• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images [50.76668288066681]
  We present a scale-aware super-resolution network to adaptively segment lesions of various sizes from low-resolution medical images. Our proposed network achieved consistent improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-30T14:25:55Z)
• Two-stage MR Image Segmentation Method for Brain Tumors based on Attention Mechanism [27.08977505280394]
  A coordination-spatial attention generation adversarial network (CASP-GAN) based on the cycle-consistent generative adversarial network (CycleGAN) is proposed. The performance of the generator is optimized by introducing the Coordinate Attention (CA) module and the Spatial Attention (SA) module. The ability to extract the structure information and the detailed information of the original medical image can help generate the desired image with higher quality.
  arXiv  Detail & Related papers  (2023-04-17T08:34:41Z)
• SuperMask: Generating High-resolution object masks from multi-view, unaligned low-resolution MRIs [0.8074019565026543]
  High-resolution isotropic MRIs are rare and typical MRIs are anisotropic, with the out-of-plane dimension having a much lower resolution. We propose a weakly-supervised deep learning-based solution to generate high-resolution masks from multiple low-resolution images.
  arXiv  Detail & Related papers  (2023-03-13T23:09:51Z)
• Large-Kernel Attention for 3D Medical Image Segmentation [14.76728117630242]
  In this paper, a novel large- kernel (LK) attention module is proposed to achieve accurate multi-organ segmentation and tumor segmentation. The advantages of convolution and self-attention are combined in the proposed LK attention module, including local contextual information, long-range dependence, and channel adaptation. The module also decomposes the LK convolution to optimize the computational cost and can be easily incorporated into FCNs such as U-Net.
  arXiv  Detail & Related papers  (2022-07-19T16:32:55Z)
• Superresolution and Segmentation of OCT scans using Multi-Stage adversarial Guided Attention Training [18.056525121226862]
  We propose the multi-stage & multi-discriminatory generative adversarial network (MultiSDGAN) to translate OCT scans in high-resolution segmentation labels. We evaluate and compare various combinations of channel and spatial attention to the MultiSDGAN architecture to extract more powerful feature maps. Our results demonstrate relative improvements of 21.44% and 19.45% on the Dice coefficient and SSIM, respectively.
  arXiv  Detail & Related papers  (2022-06-10T00:26:55Z)
• Multi-modal Aggregation Network for Fast MR Imaging [85.25000133194762]
  We propose a novel Multi-modal Aggregation Network, named MANet, which is capable of discovering complementary representations from a fully sampled auxiliary modality. In our MANet, the representations from the fully sampled auxiliary and undersampled target modalities are learned independently through a specific network. Our MANet follows a hybrid domain learning framework, which allows it to simultaneously recover the frequency signal in the $k$-space domain.
  arXiv  Detail & Related papers  (2021-10-15T13:16:59Z)
• Rethinking the Extraction and Interaction of Multi-Scale Features for Vessel Segmentation [53.187152856583396]
  We propose a novel deep learning model called PC-Net to segment retinal vessels and major arteries in 2D fundus image and 3D computed tomography angiography (CTA) scans. In PC-Net, the pyramid squeeze-and-excitation (PSE) module introduces spatial information to each convolutional block, boosting its ability to extract more effective multi-scale features.
  arXiv  Detail & Related papers  (2020-10-09T08:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.