T-FAKE: Synthesizing Thermal Images for Facial Landmarking

• URL: http://arxiv.org/abs/2408.15127v2
• Date: Fri, 4 Oct 2024 12:20:44 GMT
• Title: T-FAKE: Synthesizing Thermal Images for Facial Landmarking
• Authors: Philipp Flotho, Moritz Piening, Anna Kukleva, Gabriele Steidl,
• Abstract summary: We introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations.
• Score: 8.20594611891252
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Facial analysis is a key component in a wide range of applications such as security, autonomous driving, entertainment, and healthcare. Despite the availability of various facial RGB datasets, the thermal modality, which plays a crucial role in life sciences, medicine, and biometrics, has been largely overlooked. To address this gap, we introduce the T-FAKE dataset, a new large-scale synthetic thermal dataset with sparse and dense landmarks. To facilitate the creation of the dataset, we propose a novel RGB2Thermal loss function, which enables the transfer of thermal style to RGB faces. By utilizing the Wasserstein distance between thermal and RGB patches and the statistical analysis of clinical temperature distributions on faces, we ensure that the generated thermal images closely resemble real samples. Using RGB2Thermal style transfer based on our RGB2Thermal loss function, we create the T-FAKE dataset, a large-scale synthetic thermal dataset of faces. Leveraging our novel T-FAKE dataset, probabilistic landmark prediction, and label adaptation networks, we demonstrate significant improvements in landmark detection methods on thermal images across different landmark conventions. Our models show excellent performance with both sparse 70-point landmarks and dense 478-point landmark annotations. Our code and models are available at https://github.com/phflot/tfake.

Related papers

• CattleFace-RGBT: RGB-T Cattle Facial Landmark Benchmark [4.463254896517738]
  CattleFace-RGBT is a RGB-T Cattle Facial Landmark dataset consisting of 2,300 RGB-T image pairs, a total of 4,600 images. Applying AI to thermal images is challenging due to suboptimal results from direct thermal training and infeasible RGB-thermal alignment. We transfer models trained on RGB to thermal images and refine them using our AI-assisted annotation tool.
  arXiv  Detail & Related papers  (2024-06-05T16:29:13Z)
• ThermoNeRF: Multimodal Neural Radiance Fields for Thermal Novel View Synthesis [5.66229031510643]
  We propose ThermoNeRF, a novel approach to rendering new RGB and thermal views of a scene jointly. To overcome the lack of texture in thermal images, we use paired RGB and thermal images to learn scene density. We also introduce ThermoScenes, a new dataset to palliate the lack of available RGB+thermal datasets for scene reconstruction.
  arXiv  Detail & Related papers  (2024-03-18T18:10:34Z)
• Caltech Aerial RGB-Thermal Dataset in the Wild [14.699908177967181]
  We present the first publicly-available RGB-thermal dataset designed for aerial robotics operating in natural environments. Our dataset captures a variety of terrain across the United States, including rivers, lakes, coastlines, deserts, and forests. We provide semantic segmentation annotations for 10 classes commonly encountered in natural settings.
  arXiv  Detail & Related papers  (2024-03-13T23:31:04Z)
• Visible to Thermal image Translation for improving visual task in low light conditions [0.0]
  We have collected images from two different locations using the Parrot Anafi Thermal drone. We created a two-stream network, preprocessed, augmented, the image data, and trained the generator and discriminator models from scratch. The findings demonstrate that it is feasible to translate RGB training data to thermal data using GAN.
  arXiv  Detail & Related papers  (2023-10-31T05:18:53Z)
• Precise Facial Landmark Detection by Reference Heatmap Transformer [52.417964103227696]
  We propose a novel Reference Heatmap Transformer (RHT) for more precise facial landmark detection. The experimental results from challenging benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in the literature.
  arXiv  Detail & Related papers  (2023-03-14T12:26:48Z)
• Does Thermal Really Always Matter for RGB-T Salient Object Detection? [153.17156598262656]
  This paper proposes a network named TNet to solve the RGB-T salient object detection (SOD) task. In this paper, we introduce a global illumination estimation module to predict the global illuminance score of the image. On the other hand, we introduce a two-stage localization and complementation module in the decoding phase to transfer object localization cue and internal integrity cue in thermal features to the RGB modality.
  arXiv  Detail & Related papers  (2022-10-09T13:50:12Z)
• Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection [16.64781797503128]
  RGB-thermal object detection (RGB-T SOD) aims to locate the common prominent objects of an aligned visible and thermal infrared image pair. In this paper, we propose a novel mirror complementary Transformer network (MCNet) for RGB-T SOD. Experiments on benchmark and VT723 datasets show that the proposed method outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-07-07T20:26:09Z)
• A Synthesis-Based Approach for Thermal-to-Visible Face Verification [105.63410428506536]
  This paper presents an algorithm that achieves state-of-the-art performance on the ARL-VTF and TUFTS multi-spectral face datasets. We also present MILAB-VTF(B), a challenging multi-spectral face dataset composed of paired thermal and visible videos.
  arXiv  Detail & Related papers  (2021-08-21T17:59:56Z)
• A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset [62.193924313292875]
  We present the DEVCOM Army Research Laboratory Visible-Thermal Face dataset (ARL-VTF) With over 500,000 images from 395 subjects, the ARL-VTF dataset represents to the best of our knowledge, the largest collection of paired visible and thermal face images to date. This paper presents benchmark results and analysis on thermal face landmark detection and thermal-to-visible face verification by evaluating state-of-the-art models on the ARL-VTF dataset.
  arXiv  Detail & Related papers  (2021-01-07T17:17:12Z)
• Multi-Scale Thermal to Visible Face Verification via Attribute Guided Synthesis [55.29770222566124]
  We use attributes extracted from visible images to synthesize attribute-preserved visible images from thermal imagery for cross-modal matching. A novel multi-scale generator is proposed to synthesize the visible image from the thermal image guided by the extracted attributes. A pre-trained VGG-Face network is leveraged to extract features from the synthesized image and the input visible image for verification.
  arXiv  Detail & Related papers  (2020-04-20T01:45:05Z)
• A Transfer Learning approach to Heatmap Regression for Action Unit intensity estimation [50.261472059743845]
  Action Units (AUs) are geometrically-based atomic facial muscle movements. We propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity. A Heatmap models whether an AU occurs or not at a given spatial location.
  arXiv  Detail & Related papers  (2020-04-14T16:51:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.