Learning Representations for Masked Facial Recovery
- URL: http://arxiv.org/abs/2212.14110v1
- Date: Wed, 28 Dec 2022 22:22:15 GMT
- Title: Learning Representations for Masked Facial Recovery
- Authors: Zaigham Randhawa, Shivang Patel, Donald Adjeroh, Gianfranco Doretto
- Abstract summary: pandemic of these recent years has led to a dramatic increase in people wearing protective masks in public venues.
One way to address the problem is to revert to face recovery methods as a preprocessing step.
We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask.
- Score: 8.124282476398843
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: The pandemic of these very recent years has led to a dramatic increase in
people wearing protective masks in public venues. This poses obvious challenges
to the pervasive use of face recognition technology that now is suffering a
decline in performance. One way to address the problem is to revert to face
recovery methods as a preprocessing step. Current approaches to face
reconstruction and manipulation leverage the ability to model the face
manifold, but tend to be generic. We introduce a method that is specific for
the recovery of the face image from an image of the same individual wearing a
mask. We do so by designing a specialized GAN inversion method, based on an
appropriate set of losses for learning an unmasking encoder. With extensive
experiments, we show that the approach is effective at unmasking face images.
In addition, we also show that the identity information is preserved
sufficiently well to improve face verification performance based on several
face recognition benchmark datasets.
Related papers
- OSDFace: One-Step Diffusion Model for Face Restoration [72.5045389847792]
Diffusion models have demonstrated impressive performance in face restoration.
We propose OSDFace, a novel one-step diffusion model for face restoration.
Results demonstrate that OSDFace surpasses current state-of-the-art (SOTA) methods in both visual quality and quantitative metrics.
arXiv Detail & Related papers (2024-11-26T07:07:48Z) - Face Mask Removal with Region-attentive Face Inpainting [0.7433327915285965]
We propose a generative face inpainting method to recover/reconstruct the masked part of a face.
Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss.
We synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks.
arXiv Detail & Related papers (2024-09-10T20:10:11Z) - Seeing through the Mask: Multi-task Generative Mask Decoupling Face
Recognition [47.248075664420874]
Current general face recognition system suffers from serious performance degradation when encountering occluded scenes.
This paper proposes a Multi-task gEnerative mask dEcoupling face Recognition (MEER) network to jointly handle these two tasks.
We first present a novel mask decoupling module to disentangle mask and identity information, which makes the network obtain purer identity features from visible facial components.
arXiv Detail & Related papers (2023-11-20T03:23:03Z) - FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders [81.21440457805932]
We propose a novel framework FaceMAE, where the face privacy and recognition performance are considered simultaneously.
randomly masked face images are used to train the reconstruction module in FaceMAE.
We also perform sufficient privacy-preserving face recognition on several public face datasets.
arXiv Detail & Related papers (2022-05-23T07:19:42Z) - Restricted Black-box Adversarial Attack Against DeepFake Face Swapping [70.82017781235535]
We introduce a practical adversarial attack that does not require any queries to the facial image forgery model.
Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models.
arXiv Detail & Related papers (2022-04-26T14:36:06Z) - Mask-invariant Face Recognition through Template-level Knowledge
Distillation [3.727773051465455]
Masks affect the performance of previous face recognition systems.
We propose a mask-invariant face recognition solution (MaskInv)
In addition to the distilled knowledge, the student network benefits from additional guidance by margin-based identity classification loss.
arXiv Detail & Related papers (2021-12-10T16:19:28Z) - MLFW: A Database for Face Recognition on Masked Faces [56.441078419992046]
Masked LFW (MLFW) is a tool to generate masked faces from unmasked faces automatically.
The recognition accuracy of SOTA models declines 5%-16% on MLFW database compared with the accuracy on the original images.
arXiv Detail & Related papers (2021-09-13T09:30:10Z) - Multi-Dataset Benchmarks for Masked Identification using Contrastive
Representation Learning [0.0]
COVID-19 pandemic has drastically changed accepted norms globally.
Official documents such as passports, driving license and national identity cards are enrolled with fully uncovered face images.
In an airport or security checkpoint it is safer to match the unmasked image of the identifying document to the masked person rather than asking them to remove the mask.
We propose a contrastive visual representation learning based pre-training workflow which is specialized to masked vs unmasked face matching.
arXiv Detail & Related papers (2021-06-10T08:58:10Z) - Unmasking Face Embeddings by Self-restrained Triplet Loss for Accurate
Masked Face Recognition [6.865656740940772]
We present a solution to improve the masked face recognition performance.
Specifically, we propose the Embedding Unmasking Model (EUM) operated on top of existing face recognition models.
We also propose a novel loss function, the Self-restrained Triplet (SRT), which enabled the EUM to produce embeddings similar to these of unmasked faces of the same identities.
arXiv Detail & Related papers (2021-03-02T13:43:11Z) - Towards Face Encryption by Generating Adversarial Identity Masks [53.82211571716117]
We propose a targeted identity-protection iterative method (TIP-IM) to generate adversarial identity masks.
TIP-IM provides 95%+ protection success rate against various state-of-the-art face recognition models.
arXiv Detail & Related papers (2020-03-15T12:45:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.