MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration

• URL: http://arxiv.org/abs/2411.01399v1
• Date: Sun, 03 Nov 2024 01:30:59 GMT
• Title: MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration
• Authors: Kaiang Wen, Bin Xie, Bin Duan, Yan Yan,
• Abstract summary: Traditional learning-based approaches often consider registration networks as black boxes without interpretability. We propose MambaReg, a novel Mamba-based architecture that integrates Mamba's strong capability in capturing long sequences. Our network adeptly captures the correlation between multi-modal images, enabling focused deformation field prediction.
• Score: 13.146228081053714
• License:
• Abstract: Precise alignment of multi-modal images with inherent feature discrepancies poses a pivotal challenge in deformable image registration. Traditional learning-based approaches often consider registration networks as black boxes without interpretability. One core insight is that disentangling alignment features and non-alignment features across modalities bring benefits. Meanwhile, it is challenging for the prominent methods for image registration tasks, such as convolutional neural networks, to capture long-range dependencies by their local receptive fields. The methods often fail when the given image pair has a large misalignment due to the lack of effectively learning long-range dependencies and correspondence. In this paper, we propose MambaReg, a novel Mamba-based architecture that integrates Mamba's strong capability in capturing long sequences to address these challenges. With our proposed several sub-modules, MambaReg can effectively disentangle modality-independent features responsible for registration from modality-dependent, non-aligning features. By selectively attending to the relevant features, our network adeptly captures the correlation between multi-modal images, enabling focused deformation field prediction and precise image alignment. The Mamba-based architecture seamlessly integrates the local feature extraction power of convolutional layers with the long-range dependency modeling capabilities of Mamba. Experiments on public non-rigid RGB-IR image datasets demonstrate the superiority of our method, outperforming existing approaches in terms of registration accuracy and deformation field smoothness.

Related papers

• MatIR: A Hybrid Mamba-Transformer Image Restoration Model [95.17418386046054]
  We propose a Mamba-Transformer hybrid image restoration model called MatIR. MatIR cross-cycles the blocks of the Transformer layer and the Mamba layer to extract features. In the Mamba module, we introduce the Image Inpainting State Space (IRSS) module, which traverses along four scan paths.
  arXiv  Detail & Related papers  (2025-01-30T14:55:40Z)
• MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching [54.740256498985026]
  Keypoint detection and description methods often struggle with multimodal data. We propose a modality-invariant feature learning network (MIFNet) to compute modality-invariant features for keypoint descriptions in multimodal image matching.
  arXiv  Detail & Related papers  (2025-01-20T06:56:30Z)
• Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging [40.80197280147993]
  We propose a Mamba-inspired Joint Unfolding Network (MiJUN) to overcome the inherent nonlinear and ill-posed characteristics of HSI reconstruction. We introduce an accelerated unfolding network scheme, which reduces the reliance on initial optimization stages. We refine the scanning strategy with Mamba by integrating the tensor mode-$k$ unfolding into the Mamba network.
  arXiv  Detail & Related papers  (2025-01-02T13:56:23Z)
• Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement [54.427965535613886]
  Mamba, as a novel state-space model (SSM), has gained widespread application in natural language processing and computer vision. In this work, we introduce Mamba-SEUNet, an innovative architecture that integrates Mamba with U-Net for SE tasks.
  arXiv  Detail & Related papers  (2024-12-21T13:43:51Z)
• Multi-dimensional Visual Prompt Enhanced Image Restoration via Mamba-Transformer Aggregation [4.227991281224256]
  This paper proposes to fully utilize complementary advantages from Mamba and Transformer without sacrificing computation efficiency. The selective scanning mechanism of Mamba is employed to focus on spatial modeling, enabling capture long-range spatial dependencies. The self-attention mechanism of Transformer is applied to focus on channel modeling, avoiding high burdens that are in quadratic growth with image's spatial dimensions.
  arXiv  Detail & Related papers  (2024-12-20T12:36:34Z)
• MambaPro: Multi-Modal Object Re-Identification with Mamba Aggregation and Synergistic Prompt [60.10555128510744]
  Multi-modal object Re-IDentification (ReID) aims to retrieve specific objects by utilizing complementary image information from different modalities. Recently, large-scale pre-trained models like CLIP have demonstrated impressive performance in traditional single-modal object ReID tasks. We introduce a novel framework called MambaPro for multi-modal object ReID.
  arXiv  Detail & Related papers  (2024-12-14T06:33:53Z)
• Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentation [54.96563068182733]
  We propose Modality Adaptation with text-to-image Diffusion Models (MADM) for semantic segmentation task. MADM utilizes text-to-image diffusion models pre-trained on extensive image-text pairs to enhance the model's cross-modality capabilities. We show that MADM achieves state-of-the-art adaptation performance across various modality tasks, including images to depth, infrared, and event modalities.
  arXiv  Detail & Related papers  (2024-10-29T03:49:40Z)
• SIGMA: Selective Gated Mamba for Sequential Recommendation [56.85338055215429]
  Mamba, a recent advancement, has exhibited exceptional performance in time series prediction. We introduce a new framework named Selective Gated Mamba ( SIGMA) for Sequential Recommendation. Our results indicate that SIGMA outperforms current models on five real-world datasets.
  arXiv  Detail & Related papers  (2024-08-21T09:12:59Z)
• Self-Prior Guided Mamba-UNet Networks for Medical Image Super-Resolution [7.97504951029884]
  We propose a self-prior guided Mamba-UNet network (SMamba-UNet) for medical image super-resolution. Inspired by Mamba, our approach aims to learn the self-prior multi-scale contextual features under Mamba-UNet networks.
  arXiv  Detail & Related papers  (2024-07-08T14:41:53Z)
• RSMamba: Remote Sensing Image Classification with State Space Model [25.32283897448209]
  We introduce RSMamba, a novel architecture for remote sensing image classification. RSMamba is based on the State Space Model (SSM) and incorporates an efficient, hardware-aware design known as the Mamba. We propose a dynamic multi-path activation mechanism to augment Mamba's capacity to model non-temporal image data.
  arXiv  Detail & Related papers  (2024-03-28T17:59:49Z)
• PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition [21.761988930589727]
  PlainMamba is a simple non-hierarchical state space model (SSM) designed for general visual recognition. We adapt the selective scanning process of Mamba to the visual domain, enhancing its ability to learn features from two-dimensional images. Our architecture is designed to be easy to use and easy to scale, formed by stacking identical PlainMamba blocks.
  arXiv  Detail & Related papers  (2024-03-26T13:35:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.