Memory-Guided Collaborative Attention for Nighttime Thermal Infrared Image Colorization

• URL: http://arxiv.org/abs/2208.02960v1
• Date: Fri, 5 Aug 2022 03:04:04 GMT
• Title: Memory-Guided Collaborative Attention for Nighttime Thermal Infrared Image Colorization
• Authors: Fu-Ya Luo, Yi-Jun Cao, Kai-Fu Yang, and Yong-Jie Li
• Abstract summary: We propose a novel learning framework called Memory-guided cOllaboRative atteNtion Generative Adversarial Network (MornGAN) MornGAN is inspired by the analogical reasoning mechanisms of humans. It significantly outperforms other image-to-image translation methods in terms of semantic preservation and edge consistency.
• Score: 14.239472686466325
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nighttime thermal infrared (NTIR) image colorization, also known as translation of NTIR images into daytime color images (NTIR2DC), is a promising research direction to facilitate nighttime scene perception for humans and intelligent systems under unfavorable conditions (e.g., complete darkness). However, previously developed methods have poor colorization performance for small sample classes. Moreover, reducing the high confidence noise in pseudo-labels and addressing the problem of image gradient disappearance during translation are still under-explored, and keeping edges from being distorted during translation is also challenging. To address the aforementioned issues, we propose a novel learning framework called Memory-guided cOllaboRative atteNtion Generative Adversarial Network (MornGAN), which is inspired by the analogical reasoning mechanisms of humans. Specifically, a memory-guided sample selection strategy and adaptive collaborative attention loss are devised to enhance the semantic preservation of small sample categories. In addition, we propose an online semantic distillation module to mine and refine the pseudo-labels of NTIR images. Further, conditional gradient repair loss is introduced for reducing edge distortion during translation. Extensive experiments on the NTIR2DC task show that the proposed MornGAN significantly outperforms other image-to-image translation methods in terms of semantic preservation and edge consistency, which helps improve the object detection accuracy remarkably.

Related papers

• Retinex-RAWMamba: Bridging Demosaicing and Denoising for Low-Light RAW Image Enhancement [71.13353154514418]
  Low-light image enhancement, particularly in cross-domain tasks such as mapping from the raw domain to the sRGB domain, remains a significant challenge. We present a novel Mamba scanning mechanism, called RAWMamba, to effectively handle raw images with different CFAs. We also present a Retinex Decomposition Module (RDM) grounded in Retinex prior, which decouples illumination from reflectance to facilitate more effective denoising and automatic non-linear exposure correction.
  arXiv  Detail & Related papers  (2024-09-11T06:12:03Z)
• Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization [4.349838917565205]
  We introduce a Dense Normalization layer designed to estimate pixel-level statistical moments. This approach effectively diminishes tiling artifacts while concurrently preserving local color and hue contrasts. Our work paves the way for future exploration in handling images of arbitrary resolutions within the realm of unpaired image-to-image translation.
  arXiv  Detail & Related papers  (2024-07-05T04:14:50Z)
• You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement [50.37253008333166]
  Low-Light Image Enhancement (LLIE) task tends to restore the details and visual information from corrupted low-light images. We propose a novel trainable color space, named Horizontal/Vertical-Intensity (HVI) It not only decouples brightness and color from RGB channels to mitigate the instability during enhancement but also adapts to low-light images in different illumination ranges due to the trainable parameters.
  arXiv  Detail & Related papers  (2024-02-08T16:47:43Z)
• Nighttime Thermal Infrared Image Colorization with Feedback-based Object Appearance Learning [27.58748298687474]
  We propose a generative adversarial network incorporating feedback-based object appearance learning (FoalGAN) FoalGAN is effective for appearance learning of small objects, but also outperforms other image translation methods in terms of semantic preservation and edge consistency.
  arXiv  Detail & Related papers  (2023-10-24T09:59:55Z)
• Reverse Engineering of Imperceptible Adversarial Image Perturbations [43.87341855153572]
  We formalize the RED problem and identify a set of principles crucial to the RED approach design. We propose a new Class-Discriminative Denoising based RED framework, termed CDD-RED.
  arXiv  Detail & Related papers  (2022-03-26T19:52:40Z)
• Invertible Network for Unpaired Low-light Image Enhancement [78.33382003460903]
  We propose to leverage the invertible network to enhance low-light image in forward process and degrade the normal-light one inversely with unpaired learning. In addition to the adversarial loss, we design various loss functions to ensure the stability of training and preserve more image details. We present a progressive self-guided enhancement process for low-light images and achieve favorable performance against the SOTAs.
  arXiv  Detail & Related papers  (2021-12-24T17:00:54Z)
• LTT-GAN: Looking Through Turbulence by Inverting GANs [86.25869403782957]
  We propose the first turbulence mitigation method that makes use of visual priors encapsulated by a well-trained GAN. Based on the visual priors, we propose to learn to preserve the identity of restored images on a periodic contextual distance. Our method significantly outperforms prior art in both the visual quality and face verification accuracy of restored results.
  arXiv  Detail & Related papers  (2021-12-04T16:42:13Z)
• Thermal Infrared Image Colorization for Nighttime Driving Scenes with Top-Down Guided Attention [14.527765677864913]
  We propose a toP-down attEntion And gRadient aLignment based GAN, referred to as PearlGAN. A top-down guided attention module and an elaborate attentional loss are first designed to reduce the semantic encoding ambiguity during translation. In addition, pixel-level annotation is carried out on a subset of FLIR and KAIST datasets to evaluate the semantic preservation performance of multiple translation methods.
  arXiv  Detail & Related papers  (2021-04-29T14:35:25Z)
• Degrade is Upgrade: Learning Degradation for Low-light Image Enhancement [52.49231695707198]
  We investigate the intrinsic degradation and relight the low-light image while refining the details and color in two steps. Inspired by the color image formulation, we first estimate the degradation from low-light inputs to simulate the distortion of environment illumination color, and then refine the content to recover the loss of diffuse illumination color. Our proposed method has surpassed the SOTA by 0.95dB in PSNR on LOL1000 dataset and 3.18% in mAP on ExDark dataset.
  arXiv  Detail & Related papers  (2021-03-19T04:00:27Z)
• Context-Aware Image Denoising with Auto-Threshold Canny Edge Detection to Suppress Adversarial Perturbation [0.8021197489470756]
  This paper presents a novel context-aware image denoising algorithm. It combines an adaptive image smoothing technique and color reduction techniques to remove perturbation from adversarial images. Our results show that the proposed approach reduces adversarial perturbation in adversarial attacks and increases the robustness of the deep convolutional neural network models.
  arXiv  Detail & Related papers  (2021-01-14T19:15:28Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.