CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain

• URL: http://arxiv.org/abs/2411.16327v1
• Date: Mon, 25 Nov 2024 12:23:14 GMT
• Title: CapHDR2IR: Caption-Driven Transfer from Visible Light to Infrared Domain
• Authors: Jingchao Peng, Thomas Bashford-Rogers, Zhuang Shao, Haitao Zhao, Aru Ranjan Singh, Abhishek Goswami, Kurt Debattista,
• Abstract summary: Infrared (IR) imaging offers advantages in several fields due to its unique ability of capturing content in extreme light conditions. As an alternative, visible light can be used to synthesize IR images but this causes a loss of fidelity in image details and introduces inconsistencies due to lack of contextual awareness of the scene.
• Score: 7.007302908953179
• License:
• Abstract: Infrared (IR) imaging offers advantages in several fields due to its unique ability of capturing content in extreme light conditions. However, the demanding hardware requirements of high-resolution IR sensors limit its widespread application. As an alternative, visible light can be used to synthesize IR images but this causes a loss of fidelity in image details and introduces inconsistencies due to lack of contextual awareness of the scene. This stems from a combination of using visible light with a standard dynamic range, especially under extreme lighting, and a lack of contextual awareness can result in pseudo-thermal-crossover artifacts. This occurs when multiple objects with similar temperatures appear indistinguishable in the training data, further exacerbating the loss of fidelity. To solve this challenge, this paper proposes CapHDR2IR, a novel framework incorporating vision-language models using high dynamic range (HDR) images as inputs to generate IR images. HDR images capture a wider range of luminance variations, ensuring reliable IR image generation in different light conditions. Additionally, a dense caption branch integrates semantic understanding, resulting in more meaningful and discernible IR outputs. Extensive experiments on the HDRT dataset show that the proposed CapHDR2IR achieves state-of-the-art performance compared with existing general domain transfer methods and those tailored for visible-to-infrared image translation.

Related papers

• Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution [54.293362972473595]
  Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Current approaches to address SR tasks are either dedicated to extracting RGB image features or assuming similar degradation patterns. We propose a Contourlet refinement gate framework to restore infrared modal-specific features while preserving spectral distribution fidelity.
  arXiv  Detail & Related papers  (2024-11-19T14:24:03Z)
• HDRT: Infrared Capture for HDR Imaging [8.208995723545502]
  We propose a new approach, High Dynamic Range Thermal (HDRT), for HDR acquisition using a separate, commonly available, thermal infrared (IR) sensor. We propose a novel deep neural method (HDRTNet) which combines IR and SDR content to generate HDR images. We show substantial quantitative and qualitative quality improvements on both over- and under-exposed images, showing that our approach is robust to capturing in multiple different lighting conditions.
  arXiv  Detail & Related papers  (2024-06-08T13:43:44Z)
• NIR-Assisted Image Denoising: A Selective Fusion Approach and A Real-World Benchmark Dataset [53.79524776100983]
  Leveraging near-infrared (NIR) images to assist visible RGB image denoising shows the potential to address this issue. Existing works still struggle with taking advantage of NIR information effectively for real-world image denoising. We propose an efficient Selective Fusion Module (SFM), which can be plug-and-played into the advanced denoising networks.
  arXiv  Detail & Related papers  (2024-04-12T14:54:26Z)
• Event-based Asynchronous HDR Imaging by Temporal Incident Light Modulation [54.64335350932855]
  We propose a Pixel-Asynchronous HDR imaging system, based on key insights into the challenges in HDR imaging. Our proposed Asyn system integrates the Dynamic Vision Sensors (DVS) with a set of LCD panels. The LCD panels modulate the irradiance incident upon the DVS by altering their transparency, thereby triggering the pixel-independent event streams.
  arXiv  Detail & Related papers  (2024-03-14T13:45:09Z)
• Towards High-quality HDR Deghosting with Conditional Diffusion Models [88.83729417524823]
  High Dynamic Range (LDR) images can be recovered from several Low Dynamic Range (LDR) images by existing Deep Neural Networks (DNNs) techniques. DNNs still generate ghosting artifacts when LDR images have saturation and large motion. We formulate the HDR deghosting problem as an image generation that leverages LDR features as the diffusion model's condition.
  arXiv  Detail & Related papers  (2023-11-02T01:53:55Z)
• Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes [58.66427721308464]
  Self is a self-supervised reconstruction method that only requires dynamic multi-exposure images during training. Self achieves superior results against the state-of-the-art self-supervised methods, and comparable performance to supervised ones.
  arXiv  Detail & Related papers  (2023-10-03T07:10:49Z)
• Enhancing Low-Light Images Using Infrared-Encoded Images [81.8710581927427]
  Previous arts mainly focus on the low-light images captured in the visible spectrum using pixel-wise loss. We propose a novel approach to increase the visibility of images captured under low-light environments by removing the in-camera infrared (IR) cut-off filter.
  arXiv  Detail & Related papers  (2023-07-09T08:29:19Z)
• Visibility Constrained Wide-band Illumination Spectrum Design for Seeing-in-the-Dark [38.11468156313255]
  Seeing-in-the-dark is one of the most important and challenging computer vision tasks. In this paper, we try to robustify NIR2RGB translation by designing the optimal spectrum of auxiliary illumination in the wide-band VIS-NIR range.
  arXiv  Detail & Related papers  (2023-03-21T07:27:37Z)
• Multi-Exposure HDR Composition by Gated Swin Transformer [8.619880437958525]
  This paper provides a novel multi-exposure fusion model based on Swin Transformer. We exploit the long distance contextual dependency in the exposure-space pyramid by the self-attention mechanism. Experiments show that our model achieves the accuracy on par with current top performing multi-exposure HDR imaging models.
  arXiv  Detail & Related papers  (2023-03-15T15:38:43Z)
• Deep Snapshot HDR Imaging Using Multi-Exposure Color Filter Array [14.5106375775521]
  We introduce the idea of luminance normalization that simultaneously enables effective loss and input data normalization. Experimental results using two public HDR image datasets demonstrate that our framework outperforms other snapshot methods.
  arXiv  Detail & Related papers  (2020-11-20T06:31:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.