Global and Local Mamba Network for Multi-Modality Medical Image Super-Resolution

• URL: http://arxiv.org/abs/2504.10105v1
• Date: Mon, 14 Apr 2025 11:14:24 GMT
• Title: Global and Local Mamba Network for Multi-Modality Medical Image Super-Resolution
• Authors: Zexin Ji, Beiji Zou, Xiaoyan Kui, Sebastien Thureau, Su Ruan,
• Abstract summary: We propose a global and local Mamba network (GLMamba) for multi-modality medical image super-resolution.<n>The global Mamba branch captures long-range relationships in low-resolution inputs, and the local Mamba branch focuses more on short-range details in high-resolution reference images.<n>A modulator is designed to further enhance deformable features in both global and local Mamba blocks.
• Score: 6.646855815619148
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Convolutional neural networks and Transformer have made significant progresses in multi-modality medical image super-resolution. However, these methods either have a fixed receptive field for local learning or significant computational burdens for global learning, limiting the super-resolution performance. To solve this problem, State Space Models, notably Mamba, is introduced to efficiently model long-range dependencies in images with linear computational complexity. Relying on the Mamba and the fact that low-resolution images rely on global information to compensate for missing details, while high-resolution reference images need to provide more local details for accurate super-resolution, we propose a global and local Mamba network (GLMamba) for multi-modality medical image super-resolution. To be specific, our GLMamba is a two-branch network equipped with a global Mamba branch and a local Mamba branch. The global Mamba branch captures long-range relationships in low-resolution inputs, and the local Mamba branch focuses more on short-range details in high-resolution reference images. We also use the deform block to adaptively extract features of both branches to enhance the representation ability. A modulator is designed to further enhance deformable features in both global and local Mamba blocks. To fully integrate the reference image for low-resolution image super-resolution, we further develop a multi-modality feature fusion block to adaptively fuse features by considering similarities, differences, and complementary aspects between modalities. In addition, a contrastive edge loss (CELoss) is developed for sufficient enhancement of edge textures and contrast in medical images.

Related papers

• Prompt-Guided Dual-Path UNet with Mamba for Medical Image Segmentation [18.060052357308763]
  We propose a prompt-guided CNN-Mamba dual-path UNet, termed PGM-UNet, for medical image segmentation.<n>We introduce a prompt-guided residual Mamba module that adaptively extracts dynamic visual prompts from the original input data.<n>We also design a local-global information fusion network, comprising a local information extraction module, a prompt-guided residual Mamba module, and a multi-focus attention fusion module.
  arXiv  Detail & Related papers  (2025-03-25T12:12:07Z)
• MatIR: A Hybrid Mamba-Transformer Image Restoration Model [95.17418386046054]
  We propose a Mamba-Transformer hybrid image restoration model called MatIR. MatIR cross-cycles the blocks of the Transformer layer and the Mamba layer to extract features. In the Mamba module, we introduce the Image Inpainting State Space (IRSS) module, which traverses along four scan paths.
  arXiv  Detail & Related papers  (2025-01-30T14:55:40Z)
• Global Semantic-Guided Sub-image Feature Weight Allocation in High-Resolution Large Vision-Language Models [50.98559225639266]
  Sub-images with higher semantic relevance to the entire image encapsulate richer visual information for preserving the model's visual understanding ability. Global Semantic-guided Weight Allocator (GSWA) module allocates weights to sub-images based on their relative information density. SleighVL, a lightweight yet high-performing model, outperforms models with comparable parameters and remains competitive with larger models.
  arXiv  Detail & Related papers  (2025-01-24T06:42:06Z)
• LaMamba-Diff: Linear-Time High-Fidelity Diffusion Models Based on Local Attention and Mamba [54.85262314960038]
  Local Attentional Mamba blocks capture both global contexts and local details with linear complexity. Our model exhibits exceptional scalability and surpasses the performance of DiT across various model scales on ImageNet at 256x256 resolution. Compared to state-of-the-art diffusion models on ImageNet 256x256 and 512x512, our largest model presents notable advantages, such as a reduction of up to 62% GFLOPs.
  arXiv  Detail & Related papers  (2024-08-05T16:39:39Z)
• INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model [71.50973774576431]
  We propose a novel MLLM, INF-LLaVA, designed for effective high-resolution image perception. We introduce a Dual-perspective Cropping Module (DCM), which ensures that each sub-image contains continuous details from a local perspective. Second, we introduce Dual-perspective Enhancement Module (DEM) to enable the mutual enhancement of global and local features.
  arXiv  Detail & Related papers  (2024-07-23T06:02:30Z)
• Self-Prior Guided Mamba-UNet Networks for Medical Image Super-Resolution [7.97504951029884]
  We propose a self-prior guided Mamba-UNet network (SMamba-UNet) for medical image super-resolution. Inspired by Mamba, our approach aims to learn the self-prior multi-scale contextual features under Mamba-UNet networks.
  arXiv  Detail & Related papers  (2024-07-08T14:41:53Z)
• Deform-Mamba Network for MRI Super-Resolution [7.97504951029884]
  We propose a new architecture, called Deform-Mamba, for MR image super-resolution. We develop a Deform-Mamba encoder which is composed of two branches, modulated deform block and vision Mamba block. Thanks to the extracted features of the encoder, which include content-adaptive local and efficient global information, the vision Mamba decoder finally generates high-quality MR images.
  arXiv  Detail & Related papers  (2024-07-08T14:07:26Z)
• FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba [19.761723108363796]
  FusionMamba aims to overcome the challenges faced by CNNs and Vision Transformers (ViTs) in computer vision tasks.<n>The framework improves the visual state-space model Mamba by integrating dynamic convolution and channel attention mechanisms.<n>Experiments show that FusionMamba achieves state-of-the-art performance in a variety of multimodal image fusion tasks as well as downstream experiments.
  arXiv  Detail & Related papers  (2024-04-15T06:37:21Z)
• LKM-UNet: Large Kernel Vision Mamba UNet for Medical Image Segmentation [9.862277278217045]
  In this paper, we introduce a Large Kernel Vision Mamba U-shape Network, or LKM-UNet, for medical image segmentation. A distinguishing feature of our LKM-UNet is its utilization of large Mamba kernels, excelling in locally spatial modeling compared to small kernel-based CNNs and Transformers. Comprehensive experiments demonstrate the feasibility and the effectiveness of using large-size Mamba kernels to achieve large receptive fields.
  arXiv  Detail & Related papers  (2024-03-12T05:34:51Z)
• Scale-aware Super-resolution Network with Dual Affinity Learning for Lesion Segmentation from Medical Images [50.76668288066681]
  We present a scale-aware super-resolution network to adaptively segment lesions of various sizes from low-resolution medical images. Our proposed network achieved consistent improvements compared to other state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-30T14:25:55Z)
• InfinityGAN: Towards Infinite-Resolution Image Synthesis [92.40782797030977]
  We present InfinityGAN, a method to generate arbitrary-resolution images. We show how it trains and infers patch-by-patch seamlessly with low computational resources.
  arXiv  Detail & Related papers  (2021-04-08T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.