A Conditional U-Net Pipeline with Pre- and Post-Processing for Aerial RGB-to-Thermal Image Translation

• URL: http://arxiv.org/abs/2605.17564v1
• Date: Sun, 17 May 2026 17:49:01 GMT
• Title: A Conditional U-Net Pipeline with Pre- and Post-Processing for Aerial RGB-to-Thermal Image Translation
• Abstract summary: We propose a conditional U-Net that incorporates weather data at the bottleneck layer, complemented by targeted preprocessing and post-processing techniques.<n>We utilize a training set of 612 paired RGB and thermal images, and evaluate over 5-fold cross-validation, ultimately testing on a held-out test set.<n>Our conditional U-Net model performed best, with a peak signal-to-noise ratio (PSNR) of 14.5485, structural similarity index measure (SSIM) of 0.8095, and learned perceptual image patch similarity (LPIPS) of 0.1666.
• Score: 0.21964656111848832
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Paired RGB-thermal data has shown significant utility across a range of applications, including image fusion, object tracking, and anomaly detection; however, its broader adoption is constrained by the limited availability of aligned RGB-thermal image pairs. RGB-to-thermal (and vice versa) image translation has emerged as a practical solution to this challenge. Prior approaches including conditional generative adversarial networks (cGANs) such as ThermalGAN and Scalable Interpolant Transformer (SiT)-based architectures such as ThermalGen have demonstrated strong potential for aerial-to-thermal image translation. In this work, we explore alternative architectures that prioritize simplicity while maintaining performance. Specifically, we propose a conditional U-Net that incorporates weather data at the bottleneck layer, complemented by targeted preprocessing and post-processing techniques applied within the Pix2Pix GAN architecture. We utilize a training set of 612 paired RGB and thermal images, and evaluate over 5-fold cross-validation, ultimately testing on a held-out test set. Our conditional U-Net model performed best, with a peak signal-to-noise ratio (PSNR) of 14.5485, structural similarity index measure (SSIM) of 0.8095, and learned perceptual image patch similarity (LPIPS) of 0.1666. These results outperformed the base ThermalGen model, which attained PSNR, SSIM, and LPIPS scores of 7.56, 0.2444, and 0.6317 respectively. We find that while saturation boost and contrast enhancement for preprocessing and Gaussian blur for post-processing provide observable improvements, the incorporation of conditioning data was most effective. Our findings cement the potential of integrating auxiliary metadata into thermal image generation, suggesting that such information can serve as a proxy for environmental conditions critical to accurate thermal reconstruction.

Related papers

• Enhancing Infrared Vision: Progressive Prompt Fusion Network and Benchmark [58.61079960074608]
  Existing infrared image enhancement methods focus on tackling individual degradations.<n>All-in-one enhancement methods, commonly applied to RGB sensors, often demonstrate limited effectiveness.
  arXiv  Detail & Related papers  (2025-10-10T12:55:54Z)
• ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation [14.108149959967095]
  Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks.<n>To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution.<n>We propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation.
  arXiv  Detail & Related papers  (2025-09-29T14:55:51Z)
• T-FAKE: Synthesizing Thermal Images for Facial Landmarking [8.20594611891252]
  We introduce the T-FAKE dataset, a large-scale synthetic thermal dataset with sparse and dense landmarks.<n>We propose a novel RGB2Thermal loss function, which enables the domain-adaptive transfer of RGB faces to thermal style.<n>Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations.
  arXiv  Detail & Related papers  (2024-08-27T15:07:58Z)
• ThermoNeRF: Joint RGB and Thermal Novel View Synthesis for Building Facades using Multimodal Neural Radiance Fields [5.66229031510643]
  Thermal scene reconstruction holds great potential for various applications, such as analyzing building energy consumption and performing non-destructive infrastructure testing.<n>Existing methods typically require dense scene measurements and often rely on RGB images for 3D geometry reconstruction, projecting thermal information post-reconstruction.<n>We propose ThermoNeRF, a novel approach based on Neural Radiance Fields that jointly renders new RGB and thermal views of a scene, and ThermoScenes, a dataset of paired RGB+thermal images comprising 8 scenes of building facades and 8 scenes of everyday objects.
  arXiv  Detail & Related papers  (2024-03-18T18:10:34Z)
• Ternary-Type Opacity and Hybrid Odometry for RGB NeRF-SLAM [58.736472371951955]
  We introduce a ternary-type opacity (TT) model, which categorizes points on a ray intersecting a surface into three regions: before, on, and behind the surface. This enables a more accurate rendering of depth, subsequently improving the performance of image warping techniques. Our integrated approach of TT and HO achieves state-of-the-art performance on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-12-20T18:03:17Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Enhanced Sharp-GAN For Histopathology Image Synthesis [63.845552349914186]
  Histopathology image synthesis aims to address the data shortage issue in training deep learning approaches for accurate cancer detection. We propose a novel approach that enhances the quality of synthetic images by using nuclei topology and contour regularization. The proposed approach outperforms Sharp-GAN in all four image quality metrics on two datasets.
  arXiv  Detail & Related papers  (2023-01-24T17:54:01Z)
• Does Thermal Really Always Matter for RGB-T Salient Object Detection? [153.17156598262656]
  This paper proposes a network named TNet to solve the RGB-T salient object detection (SOD) task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality.
  arXiv  Detail & Related papers  (2022-10-09T13:50:12Z)
• Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection [16.64781797503128]
  RGB-thermal object detection (RGB-T SOD) aims to locate the common prominent objects of an aligned visible and thermal infrared image pair. In this paper, we propose a novel mirror complementary Transformer network (MCNet) for RGB-T SOD. Experiments on benchmark and VT723 datasets show that the proposed method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-07-07T20:26:09Z)
• Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion [78.19156040783061]
  Self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. The inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. We propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency.
  arXiv  Detail & Related papers  (2022-01-12T09:49:24Z)
• Meta-UDA: Unsupervised Domain Adaptive Thermal Object Detection using Meta-Learning [64.92447072894055]
  Infrared (IR) cameras are robust under adverse illumination and lighting conditions. We propose an algorithm meta-learning framework to improve existing UDA methods. We produce a state-of-the-art thermal detector for the KAIST and DSIAC datasets.
  arXiv  Detail & Related papers  (2021-10-07T02:28:18Z)
• Robust pedestrian detection in thermal imagery using synthesized images [39.33977680993236]
  We propose a method for improving pedestrian detection in the thermal domain using two stages. First, a generative data augmentation approach is used, then a domain adaptation method using generated data adapts an RGB pedestrian detector. Our detector achieves the best single-modality detection results on KAIST with respect to the state-of-the-art.
  arXiv  Detail & Related papers  (2021-02-03T11:08:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.