A Synthesis-Based Approach for Thermal-to-Visible Face Verification

• URL: http://arxiv.org/abs/2108.09558v1
• Date: Sat, 21 Aug 2021 17:59:56 GMT
• Title: A Synthesis-Based Approach for Thermal-to-Visible Face Verification
• Authors: Neehar Peri, Joshua Gleason, Carlos D. Castillo, Thirimachos Bourlai, Vishal M. Patel, Rama Chellappa
• Abstract summary: This paper presents an algorithm that achieves state-of-the-art performance on the ARL-VTF and TUFTS multi-spectral face datasets. We also present MILAB-VTF(B), a challenging multi-spectral face dataset composed of paired thermal and visible videos.
• Score: 105.63410428506536
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In recent years, visible-spectrum face verification systems have been shown to match expert forensic examiner recognition performance. However, such systems are ineffective in low-light and nighttime conditions. Thermal face imagery, which captures body heat emissions, effectively augments the visible spectrum, capturing discriminative facial features in scenes with limited illumination. Due to the increased cost and difficulty of obtaining diverse, paired thermal and visible spectrum datasets, algorithms and large-scale benchmarks for low-light recognition are limited. This paper presents an algorithm that achieves state-of-the-art performance on both the ARL-VTF and TUFTS multi-spectral face datasets. Importantly, we study the impact of face alignment, pixel-level correspondence, and identity classification with label smoothing for multi-spectral face synthesis and verification. We show that our proposed method is widely applicable, robust, and highly effective. In addition, we show that the proposed method significantly outperforms face frontalization methods on profile-to-frontal verification. Finally, we present MILAB-VTF(B), a challenging multi-spectral face dataset that is composed of paired thermal and visible videos. To the best of our knowledge, with face data from 400 subjects, this dataset represents the most extensive collection of publicly available indoor and long-range outdoor thermal-visible face imagery. Lastly, we show that our end-to-end thermal-to-visible face verification system provides strong performance on the MILAB-VTF(B) dataset.

Related papers

• Learning Domain and Pose Invariance for Thermal-to-Visible Face Recognition [6.454199265634863]
  We propose a novel Domain and Pose Invariant Framework that simultaneously learns domain and pose invariant representations. Our proposed framework is composed of modified networks for extracting the most correlated intermediate representations from off-pose thermal and frontal visible face imagery. Although DPIF focuses on learning to match off-pose thermal to frontal visible faces, we also show that DPIF enhances performance when matching frontal thermal face images to frontal visible face images.
  arXiv  Detail & Related papers  (2022-11-17T05:24:02Z)
• Simultaneous Face Hallucination and Translation for Thermal to Visible Face Verification using Axial-GAN [74.22129648654783]
  We introduce the task of thermal-to-visible face verification from low-resolution thermal images. We propose Axial-Generative Adversarial Network (Axial-GAN) to synthesize high-resolution visible images for matching.
  arXiv  Detail & Related papers  (2021-04-13T22:34:28Z)
• Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain [88.7339322596758]
  We present a novel Spatial-Phase Shallow Learning (SPSL) method, which combines spatial image and phase spectrum to capture the up-sampling artifacts of face forgery. SPSL can achieve the state-of-the-art performance on cross-datasets evaluation as well as multi-class classification and obtain comparable results on single dataset evaluation.
  arXiv  Detail & Related papers  (2021-03-02T16:45:08Z)
• A Large-Scale, Time-Synchronized Visible and Thermal Face Dataset [62.193924313292875]
  We present the DEVCOM Army Research Laboratory Visible-Thermal Face dataset (ARL-VTF) With over 500,000 images from 395 subjects, the ARL-VTF dataset represents to the best of our knowledge, the largest collection of paired visible and thermal face images to date. This paper presents benchmark results and analysis on thermal face landmark detection and thermal-to-visible face verification by evaluating state-of-the-art models on the ARL-VTF dataset.
  arXiv  Detail & Related papers  (2021-01-07T17:17:12Z)
• Recurrent Exposure Generation for Low-Light Face Detection [113.25331155337759]
  We propose a novel Recurrent Exposure Generation (REG) module and a Multi-Exposure Detection (MED) module. REG produces progressively and efficiently intermediate images corresponding to various exposure settings. Such pseudo-exposures are then fused by MED to detect faces across different lighting conditions.
  arXiv  Detail & Related papers  (2020-07-21T17:30:51Z)
• Multi-spectral Facial Landmark Detection [10.009879315990133]
  We propose a robust neural network enabled facial landmark detection, namely Deep Multi-Spectral Learning (DMSL) DMSL consists of two sub-models, i.e. face boundary detection, and landmark coordinates detection. Experiment conducted on Eurecom's visible and thermal paired database shows the superior performance of DMSL over the state-of-the-art for thermal facial landmark detection.
  arXiv  Detail & Related papers  (2020-06-09T11:43:46Z)
• Multi-Scale Thermal to Visible Face Verification via Attribute Guided Synthesis [55.29770222566124]
  We use attributes extracted from visible images to synthesize attribute-preserved visible images from thermal imagery for cross-modal matching. A novel multi-scale generator is proposed to synthesize the visible image from the thermal image guided by the extracted attributes. A pre-trained VGG-Face network is leveraged to extract features from the synthesized image and the input visible image for verification.
  arXiv  Detail & Related papers  (2020-04-20T01:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.