Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion

• URL: http://arxiv.org/abs/2201.04387v1
• Date: Wed, 12 Jan 2022 09:49:24 GMT
• Title: Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motion
• Authors: Ukcheol Shin, Kyunghyun Lee, Byeong-Uk Lee, In So Kweon
• Abstract summary: Self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. The inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. We propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency.
• Score: 78.19156040783061
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recently, self-supervised learning of depth and ego-motion from thermal images shows strong robustness and reliability under challenging scenarios. However, the inherent thermal image properties such as weak contrast, blurry edges, and noise hinder to generate effective self-supervision from thermal images. Therefore, most research relies on additional self-supervision sources such as well-lit RGB images, generative models, and Lidar information. In this paper, we conduct an in-depth analysis of thermal image characteristics that degenerates self-supervision from thermal images. Based on the analysis, we propose an effective thermal image mapping method that significantly increases image information, such as overall structure, contrast, and details, while preserving temporal consistency. The proposed method shows outperformed depth and pose results than previous state-of-the-art networks without leveraging additional RGB guidance.

Related papers

• One-step Generative Diffusion for Realistic Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called One-Step Image Rescaling Diffusion (OSIRDiff) for extreme image rescaling. OSIRDiff performs rescaling operations in the latent space of a pre-trained autoencoder. It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Data Generation Scheme for Thermal Modality with Edge-Guided Adversarial Conditional Diffusion Model [10.539491614216839]
  This paper introduces a novel approach termed the edge guided conditional diffusion model. It aims to produce meticulously aligned pseudo thermal images at the pixel level,leveraging edge information extracted from visible images. experiments on LLVIP demonstrate ECDM s superiority over existing state-of-the-art approaches in terms of image generation quality.
  arXiv  Detail & Related papers  (2024-08-07T13:01:10Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Thermal to Visible Image Synthesis under Atmospheric Turbulence [67.99407460140263]
  In biometrics and surveillance, thermal imagining modalities are often used to capture images in low-light and nighttime conditions. Such imaging systems often suffer from atmospheric turbulence, which introduces severe blur and deformation artifacts to the captured images. An end-to-end reconstruction method is proposed which can directly transform thermal images into visible-spectrum images.
  arXiv  Detail & Related papers  (2022-04-06T19:47:41Z)
• Photothermal-SR-Net: A Customized Deep Unfolding Neural Network for Photothermal Super Resolution Imaging [9.160910754837756]
  Photothermal-SR-Net is proposed in this paper, which performs deconvolution by deep unfolding considering the underlying physics. Photothermal-SR-Net applies trained block-sparsity thresholding to the acquired thermal images in each convolutional layer.
  arXiv  Detail & Related papers  (2021-04-21T14:41:04Z)
• Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN [74.22129648654783]
  We introduce the task of thermal-to-visible face verification from low-resolution thermal images. We propose Axial-Generative Adversarial Network (Axial-GAN) to synthesize high-resolution visible images for matching.
  arXiv  Detail & Related papers  (2021-04-13T22:34:28Z)
• Learning Flow-based Feature Warping for Face Frontalization with Illumination Inconsistent Supervision [73.18554605744842]
  Flow-based Feature Warping Model (FFWM) learns to synthesize photo-realistic and illumination preserving frontal images. An Illumination Preserving Module (IPM) is proposed to learn illumination preserving image synthesis. A Warp Attention Module (WAM) is introduced to reduce the pose discrepancy in the feature level.
  arXiv  Detail & Related papers  (2020-08-16T06:07:00Z)
• Exploring Thermal Images for Object Detection in Underexposure Regions for Autonomous Driving [67.69430435482127]
  Underexposure regions are vital to construct a complete perception of the surroundings for safe autonomous driving. The availability of thermal cameras has provided an essential alternate to explore regions where other optical sensors lack in capturing interpretable signals. This work proposes a domain adaptation framework which employs a style transfer technique for transfer learning from visible spectrum images to thermal images.
  arXiv  Detail & Related papers  (2020-06-01T09:59:09Z)
• Bayesian Fusion for Infrared and Visible Images [26.64101343489016]
  In this paper, a novel Bayesian fusion model is established for infrared and visible images. We aim at making the fused image satisfy human visual system. Compared with the previous methods, the novel model can generate better fused images with high-light targets and rich texture details.
  arXiv  Detail & Related papers  (2020-05-12T14:57:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.