Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention

• URL: http://arxiv.org/abs/2104.14374v1
• Date: Thu, 29 Apr 2021 14:35:25 GMT
• Title: Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention
• Authors: Fuya Luo, Yunhan Li, Guang Zeng, Peng Peng, Gang Wang, and Yongjie Li
• Abstract summary: We propose a toP-down attEntion And gRadient aLignment based GAN, referred to as PearlGAN. A top-down guided attention module and an elaborate attentional loss are first designed to reduce the semantic encoding ambiguity during translation. In addition, pixel-level annotation is carried out on a subset of FLIR and KAIST datasets to evaluate the semantic preservation performance of multiple translation methods.
• Score: 14.527765677864913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Benefitting from insensitivity to light and high penetration of foggy environments, infrared cameras are widely used for sensing in nighttime traffic scenes. However, the low contrast and lack of chromaticity of thermal infrared (TIR) images hinder the human interpretation and portability of high-level computer vision algorithms. Colorization to translate a nighttime TIR image into a daytime color (NTIR2DC) image may be a promising way to facilitate nighttime scene perception. Despite recent impressive advances in image translation, semantic encoding entanglement and geometric distortion in the NTIR2DC task remain under-addressed. Hence, we propose a toP-down attEntion And gRadient aLignment based GAN, referred to as PearlGAN. A top-down guided attention module and an elaborate attentional loss are first designed to reduce the semantic encoding ambiguity during translation. Then, a structured gradient alignment loss is introduced to encourage edge consistency between the translated and input images. In addition, pixel-level annotation is carried out on a subset of FLIR and KAIST datasets to evaluate the semantic preservation performance of multiple translation methods. Furthermore, a new metric is devised to evaluate the geometric consistency in the translation process. Extensive experiments demonstrate the superiority of the proposed PearlGAN over other image translation methods for the NTIR2DC task. The source code and labeled segmentation masks will be available at \url{https://github.com/FuyaLuo/PearlGAN/}.

Related papers

• Exploring Reliable Matching with Phase Enhancement for Night-time Semantic Segmentation [58.180226179087086]
  We propose a novel end-to-end optimized approach, named NightFormer, tailored for night-time semantic segmentation. Specifically, we design a pixel-level texture enhancement module to acquire texture-aware features hierarchically with phase enhancement and amplified attention. Our proposed method performs favorably against state-of-the-art night-time semantic segmentation methods.
  arXiv  Detail & Related papers  (2024-08-25T13:59:31Z)
• You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement [50.37253008333166]
  Low-Light Image Enhancement (LLIE) task tends to restore the details and visual information from corrupted low-light images. We propose a novel trainable color space, named Horizontal/Vertical-Intensity (HVI) It not only decouples brightness and color from RGB channels to mitigate the instability during enhancement but also adapts to low-light images in different illumination ranges due to the trainable parameters.
  arXiv  Detail & Related papers  (2024-02-08T16:47:43Z)
• Multi-scale Progressive Feature Embedding for Accurate NIR-to-RGB Spectral Domain Translation [6.580484964018551]
  We introduce a domain translation module that translates NIR source images into the grayscale target domain. By incorporating a progressive training strategy, the statistical and semantic knowledge from both task domains are efficiently aligned. Experiments show that our MPFNet outperforms state-of-the-art counterparts by 2.55 dB in the NIR-to-RGB spectral domain translation task.
  arXiv  Detail & Related papers  (2023-12-26T13:07:45Z)
• Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning [27.58748298687474]
  We propose a generative adversarial network incorporating feedback-based object appearance learning (FoalGAN) FoalGAN is effective for appearance learning of small objects, but also outperforms other image translation methods in terms of semantic preservation and edge consistency.
  arXiv  Detail & Related papers  (2023-10-24T09:59:55Z)
• Disentangled Contrastive Image Translation for Nighttime Surveillance [87.03178320662592]
  Nighttime surveillance suffers from degradation due to poor illumination and arduous human annotations. Existing methods rely on multi-spectral images to perceive objects in the dark, which are troubled by low resolution and color absence. We argue that the ultimate solution for nighttime surveillance is night-to-day translation, or Night2Day. This paper contributes a new surveillance dataset called NightSuR. It includes six scenes to support the study on nighttime surveillance.
  arXiv  Detail & Related papers  (2023-07-11T06:40:27Z)
• Enhancing Low-Light Images Using Infrared-Encoded Images [81.8710581927427]
  Previous arts mainly focus on the low-light images captured in the visible spectrum using pixel-wise loss. We propose a novel approach to increase the visibility of images captured under low-light environments by removing the in-camera infrared (IR) cut-off filter.
  arXiv  Detail & Related papers  (2023-07-09T08:29:19Z)
• StawGAN: Structural-Aware Generative Adversarial Networks for Infrared Image Translation [7.098759778181621]
  We introduce a novel model that focuses on enhancing the quality of the target generation without merely colorizing it. We test our model on aerial images of the DroneVeichle dataset containing RGB-IR paired images.
  arXiv  Detail & Related papers  (2023-05-18T11:22:33Z)
• Edge-guided Multi-domain RGB-to-TIR image Translation for Training Vision Tasks with Challenging Labels [12.701191873813583]
  The insufficient number of annotated thermal infrared (TIR) image datasets hinders TIR image-based deep learning networks to have comparable performances to that of RGB. We propose a modified multidomain RGB to TIR image translation model focused on edge preservation to employ annotated RGB images with challenging labels. We have enabled the supervised learning of deep TIR image-based optical flow estimation and object detection that ameliorated in end point error by 56.5% on average and the best object detection mAP of 23.9% respectively.
  arXiv  Detail & Related papers  (2023-01-30T06:44:38Z)
• Seeing Through The Noisy Dark: Toward Real-world Low-Light Image Enhancement and Denoising [125.56062454927755]
  Real-world low-light environment usually suffer from lower visibility and heavier noise, due to insufficient light or hardware limitation. We propose a novel end-to-end method termed Real-world Low-light Enhancement & Denoising Network (RLED-Net)
  arXiv  Detail & Related papers  (2022-10-02T14:57:23Z)
• Memory-Guided Collaborative Attention for Nighttime Thermal Infrared Image Colorization [14.239472686466325]
  We propose a novel learning framework called Memory-guided cOllaboRative atteNtion Generative Adversarial Network (MornGAN) MornGAN is inspired by the analogical reasoning mechanisms of humans. It significantly outperforms other image-to-image translation methods in terms of semantic preservation and edge consistency.
  arXiv  Detail & Related papers  (2022-08-05T03:04:04Z)
• Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement [52.49231695707198]
  We investigate the intrinsic degradation and relight the low-light image while refining the details and color in two steps. Inspired by the color image formulation, we first estimate the degradation from low-light inputs to simulate the distortion of environment illumination color, and then refine the content to recover the loss of diffuse illumination color. Our proposed method has surpassed the SOTA by 0.95dB in PSNR on LOL1000 dataset and 3.18% in mAP on ExDark dataset.
  arXiv  Detail & Related papers  (2021-03-19T04:00:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.