Data Generation Scheme for Thermal Modality with Edge-Guided Adversarial Conditional Diffusion Model

• URL: http://arxiv.org/abs/2408.03748v1
• Date: Wed, 7 Aug 2024 13:01:10 GMT
• Title: Data Generation Scheme for Thermal Modality with Edge-Guided Adversarial Conditional Diffusion Model
• Authors: Guoqing Zhu, Honghu Pan, Qiang Wang, Chao Tian, Chao Yang, Zhenyu He,
• Abstract summary: This paper introduces a novel approach termed the edge guided conditional diffusion model. It aims to produce meticulously aligned pseudo thermal images at the pixel level,leveraging edge information extracted from visible images. experiments on LLVIP demonstrate ECDM s superiority over existing state-of-the-art approaches in terms of image generation quality.
• Score: 10.539491614216839
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In challenging low light and adverse weather conditions,thermal vision algorithms,especially object detection,have exhibited remarkable potential,contrasting with the frequent struggles encountered by visible vision algorithms. Nevertheless,the efficacy of thermal vision algorithms driven by deep learning models remains constrained by the paucity of available training data samples. To this end,this paper introduces a novel approach termed the edge guided conditional diffusion model. This framework aims to produce meticulously aligned pseudo thermal images at the pixel level,leveraging edge information extracted from visible images. By utilizing edges as contextual cues from the visible domain,the diffusion model achieves meticulous control over the delineation of objects within the generated images. To alleviate the impacts of those visible-specific edge information that should not appear in the thermal domain,a two-stage modality adversarial training strategy is proposed to filter them out from the generated images by differentiating the visible and thermal modality. Extensive experiments on LLVIP demonstrate ECDM s superiority over existing state-of-the-art approaches in terms of image generation quality.

Related papers

• Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theory [19.205929427075965]
  We conceptualize the diffusion model as a black-box image render and strategically decompose its energy function in alignment with the image formation model. It generates images with realistic illumination effects, including cast shadow, soft shadow, and inter-reflections.
  arXiv  Detail & Related papers  (2024-07-29T03:15:07Z)
• Inhomogeneous illumination image enhancement under ex-tremely low visibility condition [3.534798835599242]
  Imaging through dense fog presents unique challenges, with essential visual information crucial for applications like object detection and recognition obscured, thereby hindering conventional image processing methods. We introduce in this paper a novel method that adaptively filters background illumination based on Structural Differential and Integral Filtering (F) to enhance only vital signal information. Our findings demonstrate that our proposed method significantly enhances signal clarity under extremely low visibility conditions and out-performs existing techniques, offering substantial improvements for deep fog imaging applications.
  arXiv  Detail & Related papers  (2024-04-26T16:09:42Z)
• Image Inpainting via Tractable Steering of Diffusion Models [54.13818673257381]
  This paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior. Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs) We show that our approach can consistently improve the overall quality and semantic coherence of inpainted images with only 10% additional computational overhead.
  arXiv  Detail & Related papers  (2023-11-28T21:14:02Z)
• Thermal to Visible Image Synthesis under Atmospheric Turbulence [67.99407460140263]
  In biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. Such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. An end-to-end reconstruction method is proposed which can directly transform thermal images into visible-spectrum images.
  arXiv  Detail & Related papers  (2022-04-06T19:47:41Z)
• Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion [78.19156040783061]
  Self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. The inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. We propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency.
  arXiv  Detail & Related papers  (2022-01-12T09:49:24Z)
• A Synthesis-Based Approach for Thermal-to-Visible Face Verification [105.63410428506536]
  This paper presents an algorithm that achieves state-of-the-art performance on the ARL-VTF and TUFTS multi-spectral face datasets. We also present MILAB-VTF(B), a challenging multi-spectral face dataset composed of paired thermal and visible videos.
  arXiv  Detail & Related papers  (2021-08-21T17:59:56Z)
• Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN [74.22129648654783]
  We introduce the task of thermal-to-visible face verification from low-resolution thermal images. We propose Axial-Generative Adversarial Network (Axial-GAN) to synthesize high-resolution visible images for matching.
  arXiv  Detail & Related papers  (2021-04-13T22:34:28Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)
• Unsupervised Image-generation Enhanced Adaptation for Object Detection in Thermal images [4.810743887667828]
  This paper proposes an unsupervised image-generation enhanced adaptation method for object detection in thermal images. To reduce the gap between visible domain and thermal domain, the proposed method manages to generate simulated fake thermal images. Experiments demonstrate the effectiveness and superiority of the proposed method.
  arXiv  Detail & Related papers  (2020-02-17T04:53:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.