Implicit Multi-Spectral Transformer: An Lightweight and Effective Visible to Infrared Image Translation Model

• URL: http://arxiv.org/abs/2404.07072v2
• Date: Sat, 27 Apr 2024 07:39:45 GMT
• Title: Implicit Multi-Spectral Transformer: An Lightweight and Effective Visible to Infrared Image Translation Model
• Authors: Yijia Chen, Pinghua Chen, Xiangxin Zhou, Yingtie Lei, Ziyang Zhou, Mingxian Li,
• Abstract summary: In computer vision, visible light images often exhibit low contrast in low-light conditions, presenting a significant challenge. Recent advancements in deep learning, particularly the deployment of Generative Adversarial Networks (GANs), have facilitated the transformation of visible light images to infrared images. We propose a novel end-to-end Transformer-based model that efficiently converts visible light images into high-fidelity infrared images.
• Score: 0.6817102408452475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the field of computer vision, visible light images often exhibit low contrast in low-light conditions, presenting a significant challenge. While infrared imagery provides a potential solution, its utilization entails high costs and practical limitations. Recent advancements in deep learning, particularly the deployment of Generative Adversarial Networks (GANs), have facilitated the transformation of visible light images to infrared images. However, these methods often experience unstable training phases and may produce suboptimal outputs. To address these issues, we propose a novel end-to-end Transformer-based model that efficiently converts visible light images into high-fidelity infrared images. Initially, the Texture Mapping Module and Color Perception Adapter collaborate to extract texture and color features from the visible light image. The Dynamic Fusion Aggregation Module subsequently integrates these features. Finally, the transformation into an infrared image is refined through the synergistic action of the Color Perception Adapter and the Enhanced Perception Attention mechanism. Comprehensive benchmarking experiments confirm that our model outperforms existing methods, producing infrared images of markedly superior quality, both qualitatively and quantitatively. Furthermore, the proposed model enables more effective downstream applications for infrared images than other methods.

Related papers

• LTCF-Net: A Transformer-Enhanced Dual-Channel Fourier Framework for Low-Light Image Restoration [1.049712834719005]
  We introduce LTCF-Net, a novel network architecture designed for enhancing low-light images. Our approach utilizes two color spaces - LAB and YUV - to efficiently separate and process color information. Our model incorporates the Transformer architecture to comprehensively understand image content.
  arXiv  Detail & Related papers  (2024-11-24T07:21:17Z)
• Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution [54.293362972473595]
  Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Current approaches to address SR tasks are either dedicated to extracting RGB image features or assuming similar degradation patterns. We propose a Contourlet refinement gate framework to restore infrared modal-specific features while preserving spectral distribution fidelity.
  arXiv  Detail & Related papers  (2024-11-19T14:24:03Z)
• Infrared-Assisted Single-Stage Framework for Joint Restoration and Fusion of Visible and Infrared Images under Hazy Conditions [9.415977819944246]
  We propose a joint learning framework that utilizes infrared image for the restoration and fusion of hazy IR-VIS images. Our method effectively fuses IR-VIS images while removing haze, yielding clear, haze-free fusion results.
  arXiv  Detail & Related papers  (2024-11-16T02:57:12Z)
• PID: Physics-Informed Diffusion Model for Infrared Image Generation [11.416759828137701]
  Infrared imaging technology has gained significant attention for its reliable sensing ability in low visibility conditions. Most existing image translation methods treat infrared images as a stylistic variation, neglecting the underlying physical laws. We propose a Physics-Informed Diffusion (PID) model for translating RGB images to infrared images that adhere to physical laws.
  arXiv  Detail & Related papers  (2024-07-12T14:32:30Z)
• CodeEnhance: A Codebook-Driven Approach for Low-Light Image Enhancement [97.95330185793358]
  Low-light image enhancement (LLIE) aims to improve low-illumination images. Existing methods face two challenges: uncertainty in restoration from diverse brightness degradations and loss of texture and color information. We propose a novel enhancement approach, CodeEnhance, by leveraging quantized priors and image refinement.
  arXiv  Detail & Related papers  (2024-04-08T07:34:39Z)
• IAIFNet: An Illumination-Aware Infrared and Visible Image Fusion Network [13.11361803763253]
  We propose an Illumination-Aware Infrared and Visible Image Fusion Network, named as IAIFNet. In our framework, an illumination enhancement network first estimates the incident illumination maps of input images. With the help of proposed adaptive differential fusion module (ADFM) and salient target aware module (STAM), an image fusion network effectively integrates the salient features of the illumination-enhanced infrared and visible images into a fusion image of high visual quality.
  arXiv  Detail & Related papers  (2023-09-26T15:12:29Z)
• Low-Light Image Enhancement with Illumination-Aware Gamma Correction and Complete Image Modelling Network [69.96295927854042]
  Low-light environments usually lead to less informative large-scale dark areas. We propose to integrate the effectiveness of gamma correction with the strong modelling capacities of deep networks. Because exponential operation introduces high computational complexity, we propose to use Taylor Series to approximate gamma correction.
  arXiv  Detail & Related papers  (2023-08-16T08:46:51Z)
• Breaking Modality Disparity: Harmonized Representation for Infrared and Visible Image Registration [66.33746403815283]
  We propose a scene-adaptive infrared and visible image registration. We employ homography to simulate the deformation between different planes. We propose the first ground truth available misaligned infrared and visible image dataset.
  arXiv  Detail & Related papers  (2023-04-12T06:49:56Z)
• Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration [59.02821429555375]
  We present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion. To better fuse the registered infrared images and visible images, we present a feature Interaction Fusion Module (IFM)
  arXiv  Detail & Related papers  (2022-05-24T07:51:57Z)
• Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement [52.49231695707198]
  We investigate the intrinsic degradation and relight the low-light image while refining the details and color in two steps. Inspired by the color image formulation, we first estimate the degradation from low-light inputs to simulate the distortion of environment illumination color, and then refine the content to recover the loss of diffuse illumination color. Our proposed method has surpassed the SOTA by 0.95dB in PSNR on LOL1000 dataset and 3.18% in mAP on ExDark dataset.
  arXiv  Detail & Related papers  (2021-03-19T04:00:27Z)
• Bayesian Fusion for Infrared and Visible Images [26.64101343489016]
  In this paper, a novel Bayesian fusion model is established for infrared and visible images. We aim at making the fused image satisfy human visual system. Compared with the previous methods, the novel model can generate better fused images with high-light targets and rich texture details.
  arXiv  Detail & Related papers  (2020-05-12T14:57:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.