ColorMamba: Towards High-quality NIR-to-RGB Spectral Translation with Mamba

• URL: http://arxiv.org/abs/2408.08087v1
• Date: Thu, 15 Aug 2024 11:29:13 GMT
• Title: ColorMamba: Towards High-quality NIR-to-RGB Spectral Translation with Mamba
• Authors: Huiyu Zhai, Guang Jin, Xingxing Yang, Guosheng Kang,
• Abstract summary: Translating NIR to the visible spectrum is challenging due to cross-domain complexities. Current models struggle to balance a broad receptive field with computational efficiency, limiting practical use. We propose a simple but effective backbone, dubbed ColorMamba, which first introduces Mamba into spectral translation tasks.
• Score: 0.12499537119440242
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Translating NIR to the visible spectrum is challenging due to cross-domain complexities. Current models struggle to balance a broad receptive field with computational efficiency, limiting practical use. Although the Selective Structured State Space Model, especially the improved version, Mamba, excels in generative tasks by capturing long-range dependencies with linear complexity, its default approach of converting 2D images into 1D sequences neglects local context. In this work, we propose a simple but effective backbone, dubbed ColorMamba, which first introduces Mamba into spectral translation tasks. To explore global long-range dependencies and local context for efficient spectral translation, we introduce learnable padding tokens to enhance the distinction of image boundaries and prevent potential confusion within the sequence model. Furthermore, local convolutional enhancement and agent attention are designed to improve the vanilla Mamba. Moreover, we exploit the HSV color to provide multi-scale guidance in the reconstruction process for more accurate spectral translation. Extensive experiments show that our ColorMamba achieves a 1.02 improvement in terms of PSNR compared with the state-of-the-art method. Our code is available at https://github.com/AlexYangxx/ColorMamba.

Related papers

• MatIR: A Hybrid Mamba-Transformer Image Restoration Model [95.17418386046054]
  We propose a Mamba-Transformer hybrid image restoration model called MatIR. MatIR cross-cycles the blocks of the Transformer layer and the Mamba layer to extract features. In the Mamba module, we introduce the Image Inpainting State Space (IRSS) module, which traverses along four scan paths.
  arXiv  Detail & Related papers  (2025-01-30T14:55:40Z)
• DM-Mamba: Dual-domain Multi-scale Mamba for MRI reconstruction [6.341065683872316]
  Mamba is a new paradigm for long-range dependency modeling with linear complexity. Mamba's row-wise and column-wise scanning disrupts k-space's unique spectrum. Our method significantly outperforms state-of-the-art methods with lower computational cost.
  arXiv  Detail & Related papers  (2025-01-14T14:41:51Z)
• Detail Matters: Mamba-Inspired Joint Unfolding Network for Snapshot Spectral Compressive Imaging [40.80197280147993]
  We propose a Mamba-inspired Joint Unfolding Network (MiJUN) to overcome the inherent nonlinear and ill-posed characteristics of HSI reconstruction. We introduce an accelerated unfolding network scheme, which reduces the reliance on initial optimization stages. We refine the scanning strategy with Mamba by integrating the tensor mode-$k$ unfolding into the Mamba network.
  arXiv  Detail & Related papers  (2025-01-02T13:56:23Z)
• MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration [13.146228081053714]
  Traditional learning-based approaches often consider registration networks as black boxes without interpretability. We propose MambaReg, a novel Mamba-based architecture that integrates Mamba's strong capability in capturing long sequences. Our network adeptly captures the correlation between multi-modal images, enabling focused deformation field prediction.
  arXiv  Detail & Related papers  (2024-11-03T01:30:59Z)
• V2M: Visual 2-Dimensional Mamba for Image Representation Learning [68.51380287151927]
  Mamba has garnered widespread attention due to its flexible design and efficient hardware performance to process 1D sequences. Recent studies have attempted to apply Mamba to the visual domain by flattening 2D images into patches and then regarding them as a 1D sequence. We propose a Visual 2-Dimensional Mamba model as a complete solution, which directly processes image tokens in the 2D space.
  arXiv  Detail & Related papers  (2024-10-14T11:11:06Z)
• MxT: Mamba x Transformer for Image Inpainting [11.447968918063335]
  Image inpainting aims to restore missing or damaged regions of images with semantically coherent content. We introduce MxT composed of the proposed Hybrid Module (HM), which combines Mamba with the transformer in a synergistic manner. Our HM facilitates dual-level interaction learning at both pixel and patch levels, greatly enhancing the model to reconstruct images with high quality and contextual accuracy.
  arXiv  Detail & Related papers  (2024-07-23T02:21:11Z)
• DeciMamba: Exploring the Length Extrapolation Potential of Mamba [89.07242846058023]
  We introduce DeciMamba, a context-extension method specifically designed for Mamba. Experiments over real-world long-range NLP tasks show that DeciMamba can extrapolate to context lengths significantly longer than the ones seen during training.
  arXiv  Detail & Related papers  (2024-06-20T17:40:18Z)
• MiM-ISTD: Mamba-in-Mamba for Efficient Infrared Small Target Detection [72.46396769642787]
  We develop a nested structure, Mamba-in-Mamba (MiM-ISTD), for efficient infrared small target detection. MiM-ISTD is $8 times$ faster than the SOTA method and reduces GPU memory usage by 62.2$%$ when testing on $2048 times 2048$ images.
  arXiv  Detail & Related papers  (2024-03-04T15:57:29Z)
• MambaIR: A Simple Baseline for Image Restoration with State-Space Model [46.827053426281715]
  We introduce MambaIR, which introduces both local enhancement and channel attention to improve the vanilla Mamba. Our method outperforms SwinIR by up to 0.45dB on image SR, using similar computational cost but with a global receptive field.
  arXiv  Detail & Related papers  (2024-02-23T23:15:54Z)
• Low Light Image Enhancement via Global and Local Context Modeling [164.85287246243956]
  We introduce a context-aware deep network for low-light image enhancement. First, it features a global context module that models spatial correlations to find complementary cues over full spatial domain. Second, it introduces a dense residual block that captures local context with a relatively large receptive field.
  arXiv  Detail & Related papers  (2021-01-04T09:40:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.