A Bidirectional Conversion Network for Cross-Spectral Face Recognition

• URL: http://arxiv.org/abs/2205.01595v1
• Date: Tue, 3 May 2022 16:20:10 GMT
• Title: A Bidirectional Conversion Network for Cross-Spectral Face Recognition
• Authors: Zhicheng Cao, Jiaxuan Zhang, Liaojun Pang
• Abstract summary: Cross-spectral face recognition is challenging due to the dramatic difference between the visible light and IR imageries. This paper proposes a framework of bidirectional cross-spectral conversion (BCSC-GAN) between the heterogeneous face images. The network reduces the cross-spectral recognition problem into an intra-spectral problem, and improves performance by fusing bidirectional information.
• Score: 1.9766522384767227
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Face recognition in the infrared (IR) band has become an important supplement to visible light face recognition due to its advantages of independent background light, strong penetration, ability of imaging under harsh environments such as nighttime, rain and fog. However, cross-spectral face recognition (i.e., VIS to IR) is very challenging due to the dramatic difference between the visible light and IR imageries as well as the lack of paired training data. This paper proposes a framework of bidirectional cross-spectral conversion (BCSC-GAN) between the heterogeneous face images, and designs an adaptive weighted fusion mechanism based on information fusion theory. The network reduces the cross-spectral recognition problem into an intra-spectral problem, and improves performance by fusing bidirectional information. Specifically, a face identity retaining module (IRM) is introduced with the ability to preserve identity features, and a new composite loss function is designed to overcome the modal differences caused by different spectral characteristics. Two datasets of TINDERS and CASIA were tested, where performance metrics of FID, recognition rate, equal error rate and normalized distance were compared. Results show that our proposed network is superior than other state-of-the-art methods. Additionally, the proposed rule of Self Adaptive Weighted Fusion (SAWF) is better than the recognition results of the unfused case and other traditional fusion rules that are commonly used, which further justifies the effectiveness and superiority of the proposed bidirectional conversion approach.

Related papers

• Cross-Modality Perturbation Synergy Attack for Person Re-identification [66.48494594909123]
  The main challenge in cross-modality ReID lies in effectively dealing with visual differences between different modalities. Existing attack methods have primarily focused on the characteristics of the visible image modality. This study proposes a universal perturbation attack specifically designed for cross-modality ReID.
  arXiv  Detail & Related papers  (2024-01-18T15:56:23Z)
• Frequency Domain Modality-invariant Feature Learning for Visible-infrared Person Re-Identification [79.9402521412239]
  We propose a novel Frequency Domain modality-invariant feature learning framework (FDMNet) to reduce modality discrepancy from the frequency domain perspective. Our framework introduces two novel modules, namely the Instance-Adaptive Amplitude Filter (IAF) and the Phrase-Preserving Normalization (PPNorm)
  arXiv  Detail & Related papers  (2024-01-03T17:11:27Z)
• Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition [31.783671943393344]
  We give the first attempt to deep NIR facial expression recognition and proposed a novel method called near-infrared facial expression transformer (NFER-Former) NFER-Former disentangles the expression information and spectrum information from the input image, so that the expression features can be extracted without the interference of spectrum variation. We have constructed a large NIR-VIS Facial Expression dataset that includes 360 subjects to better validate the efficiency of NFER-Former.
  arXiv  Detail & Related papers  (2023-12-10T15:15:50Z)
• Exploring Invariant Representation for Visible-Infrared Person Re-Identification [77.06940947765406]
  Cross-spectral person re-identification, which aims to associate identities to pedestrians across different spectra, faces a main challenge of the modality discrepancy. In this paper, we address the problem from both image-level and feature-level in an end-to-end hybrid learning framework named robust feature mining network (RFM) Experiment results on two standard cross-spectral person re-identification datasets, RegDB and SYSU-MM01, have demonstrated state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-02-02T05:24:50Z)
• Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration [59.02821429555375]
  We present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion. To better fuse the registered infrared images and visible images, we present a feature Interaction Fusion Module (IFM)
  arXiv  Detail & Related papers  (2022-05-24T07:51:57Z)
• Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection [65.30079184700755]
  This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks. This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network.
  arXiv  Detail & Related papers  (2022-03-30T11:44:56Z)
• Deep GAN-Based Cross-Spectral Cross-Resolution Iris Recognition [15.425678759101203]
  Cross-spectral iris recognition has emerged as a promising biometric approach to establish the identity of individuals. matching iris images acquired at different spectral bands (i.e., matching a visible (VIS) iris probe to a gallery of near-infrared (NIR) iris images or vice versa) shows a significant performance degradation. We have investigated a range of deep convolutional generative adversarial network (DCGAN) architectures to further improve the accuracy of cross-spectral iris recognition methods.
  arXiv  Detail & Related papers  (2021-08-03T15:30:04Z)
• Hierarchical Deep CNN Feature Set-Based Representation Learning for Robust Cross-Resolution Face Recognition [59.29808528182607]
  Cross-resolution face recognition (CRFR) is important in intelligent surveillance and biometric forensics. Existing shallow learning-based and deep learning-based methods focus on mapping the HR-LR face pairs into a joint feature space. In this study, we desire to fully exploit the multi-level deep convolutional neural network (CNN) feature set for robust CRFR.
  arXiv  Detail & Related papers  (2021-03-25T14:03:42Z)
• Cross-Spectral Iris Matching Using Conditional Coupled GAN [22.615156512223766]
  Cross-spectral iris recognition is emerging as a promising biometric approach to authenticating the identity of individuals. matching iris images acquired at different spectral bands shows significant performance degradation when compared to single-band near-infrared (NIR) matching. We propose a conditional coupled generative adversarial network (CpGAN) architecture for cross-spectral iris recognition.
  arXiv  Detail & Related papers  (2020-10-09T19:13:24Z)
• HyperFaceNet: A Hyperspectral Face Recognition Method Based on Deep Fusion [0.7734726150561088]
  How to fuse different light bands, i.e., hyperspectral face recognition, is still an open research problem. We propose a new fusion model (termed HyperFaceNet) especially for hyperspectral faces. Our method is proved to be of higher recognition rates than face recognition using either visible light or the infrared.
  arXiv  Detail & Related papers  (2020-08-02T14:59:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.