A Unified Framework for Masked and Mask-Free Face Recognition via Feature Rectification

• URL: http://arxiv.org/abs/2202.07358v1
• Date: Tue, 15 Feb 2022 12:37:59 GMT
• Title: A Unified Framework for Masked and Mask-Free Face Recognition via Feature Rectification
• Authors: Shaozhe Hao, Chaofeng Chen, Zhenfang Chen, Kwan-Yee K. Wong
• Abstract summary: We propose a unified framework, named Face Feature Rectification Network (FFR-Net), for recognizing both masked and mask-free faces alike. We introduce rectification blocks to rectify features extracted by a state-of-the-art recognition model, in both spatial and channel dimensions. Experiments show that our framework can learn a rectified feature space for recognizing both masked and mask-free faces effectively.
• Score: 19.417191498842044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face recognition under ideal conditions is now considered a well-solved problem with advances in deep learning. Recognizing faces under occlusion, however, still remains a challenge. Existing techniques often fail to recognize faces with both the mouth and nose covered by a mask, which is now very common under the COVID-19 pandemic. Common approaches to tackle this problem include 1) discarding information from the masked regions during recognition and 2) restoring the masked regions before recognition. Very few works considered the consistency between features extracted from masked faces and from their mask-free counterparts. This resulted in models trained for recognizing masked faces often showing degraded performance on mask-free faces. In this paper, we propose a unified framework, named Face Feature Rectification Network (FFR-Net), for recognizing both masked and mask-free faces alike. We introduce rectification blocks to rectify features extracted by a state-of-the-art recognition model, in both spatial and channel dimensions, to minimize the distance between a masked face and its mask-free counterpart in the rectified feature space. Experiments show that our unified framework can learn a rectified feature space for recognizing both masked and mask-free faces effectively, achieving state-of-the-art results. Project code: https://github.com/haoosz/FFR-Net

Related papers

• Face Mask Removal with Region-attentive Face Inpainting [0.7433327915285965]
  We propose a generative face inpainting method to recover/reconstruct the masked part of a face. Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss. We synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks.
  arXiv  Detail & Related papers  (2024-09-10T20:10:11Z)
• Seeing through the Mask: Multi-task Generative Mask Decoupling Face Recognition [47.248075664420874]
  Current general face recognition system suffers from serious performance degradation when encountering occluded scenes. This paper proposes a Multi-task gEnerative mask dEcoupling face Recognition (MEER) network to jointly handle these two tasks. We first present a novel mask decoupling module to disentangle mask and identity information, which makes the network obtain purer identity features from visible facial components.
  arXiv  Detail & Related papers  (2023-11-20T03:23:03Z)
• Mask-invariant Face Recognition through Template-level Knowledge Distillation [3.727773051465455]
  Masks affect the performance of previous face recognition systems. We propose a mask-invariant face recognition solution (MaskInv) In addition to the distilled knowledge, the student network benefits from additional guidance by margin-based identity classification loss.
  arXiv  Detail & Related papers  (2021-12-10T16:19:28Z)
• FocusFace: Multi-task Contrastive Learning for Masked Face Recognition [4.420321822469077]
  SARS-CoV-2 has presented direct and indirect challenges to the scientific community. Face recognition methods struggle to perform identity verification with similar accuracy on masked and unmasked individuals. We propose FocusFace, a multi-task architecture that uses contrastive learning to be able to accurately perform masked face recognition.
  arXiv  Detail & Related papers  (2021-10-28T08:17:12Z)
• MLFW: A Database for Face Recognition on Masked Faces [56.441078419992046]
  Masked LFW (MLFW) is a tool to generate masked faces from unmasked faces automatically. The recognition accuracy of SOTA models declines 5%-16% on MLFW database compared with the accuracy on the original images.
  arXiv  Detail & Related papers  (2021-09-13T09:30:10Z)
• End2End Occluded Face Recognition by Masking Corrupted Features [82.27588990277192]
  State-of-the-art general face recognition models do not generalize well to occluded face images. This paper presents a novel face recognition method that is robust to occlusions based on a single end-to-end deep neural network. Our approach, named FROM (Face Recognition with Occlusion Masks), learns to discover the corrupted features from the deep convolutional neural networks, and clean them by the dynamically learned masks.
  arXiv  Detail & Related papers  (2021-08-21T09:08:41Z)
• FT-TDR: Frequency-guided Transformer and Top-Down Refinement Network for Blind Face Inpainting [77.78305705925376]
  Blind face inpainting refers to the task of reconstructing visual contents without explicitly indicating the corrupted regions in a face image. We propose a novel two-stage blind face inpainting method named Frequency-guided Transformer and Top-Down Refinement Network (FT-TDR) to tackle these challenges.
  arXiv  Detail & Related papers  (2021-08-10T03:12:01Z)
• Image Inpainting by End-to-End Cascaded Refinement with Mask Awareness [66.55719330810547]
  Inpainting arbitrary missing regions is challenging because learning valid features for various masked regions is nontrivial. We propose a novel mask-aware inpainting solution that learns multi-scale features for missing regions in the encoding phase. Our framework is validated both quantitatively and qualitatively via extensive experiments on three public datasets.
  arXiv  Detail & Related papers  (2021-04-28T13:17:47Z)
• Towards NIR-VIS Masked Face Recognition [47.00916333095693]
  Near-infrared to visible (NIR-VIS) face recognition is the most common case in heterogeneous face recognition. We propose a novel training method to maximize the mutual information shared by the face representation of two domains. In addition, a 3D face reconstruction based approach is employed to synthesize masked face from the existing NIR image.
  arXiv  Detail & Related papers  (2021-04-14T10:40:09Z)
• Unmasking Face Embeddings by Self-restrained Triplet Loss for Accurate Masked Face Recognition [6.865656740940772]
  We present a solution to improve the masked face recognition performance. Specifically, we propose the Embedding Unmasking Model (EUM) operated on top of existing face recognition models. We also propose a novel loss function, the Self-restrained Triplet (SRT), which enabled the EUM to produce embeddings similar to these of unmasked faces of the same identities.
  arXiv  Detail & Related papers  (2021-03-02T13:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.