A 3D model-based approach for fitting masks to faces in the wild

• URL: http://arxiv.org/abs/2103.00803v1
• Date: Mon, 1 Mar 2021 06:50:18 GMT
• Title: A 3D model-based approach for fitting masks to faces in the wild
• Authors: Je Hyeong Hong, Hanjo Kim, Minsoo Kim, Gi Pyo Nam, Junghyun Cho, Hyeong-Seok Ko, Ig-Jae Kim
• Abstract summary: We present a 3D model-based approach called WearMask3D for augmenting face images of various poses to the masked face counterparts. Our method proceeds by first fitting a 3D morphable model on the input image, second overlaying the mask surface onto the face model and warping the respective mask texture, and last projecting the 3D mask back to 2D. Experimental results demonstrate WearMask3D produces more realistic masked images, and utilizing these images for training leads to improved recognition accuracy of masked faces.
• Score: 9.958467179573235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Face recognition research now requires a large number of labelled masked face images in the era of this unprecedented COVID-19 pandemic. Unfortunately, the rapid spread of the virus has left us little time to prepare for such dataset in the wild. To circumvent this issue, we present a 3D model-based approach called WearMask3D for augmenting face images of various poses to the masked face counterparts. Our method proceeds by first fitting a 3D morphable model on the input image, second overlaying the mask surface onto the face model and warping the respective mask texture, and last projecting the 3D mask back to 2D. The mask texture is adapted based on the brightness and resolution of the input image. By working in 3D, our method can produce more natural masked faces of diverse poses from a single mask texture. To compare precisely between different augmentation approaches, we have constructed a dataset comprising masked and unmasked faces with labels called MFW-mini. Experimental results demonstrate WearMask3D, which will be made publicly available, produces more realistic masked images, and utilizing these images for training leads to improved recognition accuracy of masked faces compared to the state-of-the-art.

Related papers

• MaskRenderer: 3D-Infused Multi-Mask Realistic Face Reenactment [0.7673339435080445]
  We present a novel end-to-end identity-agnostic face reenactment system, MaskRenderer, that can generate realistic, high fidelity frames in real-time.
  arXiv  Detail & Related papers  (2023-09-10T17:41:46Z)
• HiMFR: A Hybrid Masked Face Recognition Through Face Inpainting [0.7868449549351486]
  We propose an end-to-end hybrid masked face recognition system, namely HiMFR. Masked face detector module applies a pretrained Vision Transformer to detect whether faces are covered with masked or not. Inpainting module uses a fine-tune image inpainting model based on a Generative Adversarial Network (GAN) to restore faces. Finally, the hybrid face recognition module based on ViT with an EfficientNetB3 backbone recognizes the faces.
  arXiv  Detail & Related papers  (2022-09-19T11:26:49Z)
• Shape-Aware Masking for Inpainting in Medical Imaging [49.61617087640379]
  Inpainting has been proposed as a successful deep learning technique for unsupervised medical image model discovery. We introduce a method for generating shape-aware masks for inpainting, which aims at learning the statistical shape prior. We propose an unsupervised guided masking approach based on an off-the-shelf inpainting model and a superpixel over-segmentation algorithm.
  arXiv  Detail & Related papers  (2022-07-12T18:35:17Z)
• Non-Deterministic Face Mask Removal Based On 3D Priors [3.8502825594372703]
  The proposed approach integrates a multi-task 3D face reconstruction module with a face inpainting module. By gradually controlling the 3D shape parameters, our method generates high-quality dynamic inpainting results with different expressions and mouth movements.
  arXiv  Detail & Related papers  (2022-02-20T16:27:44Z)
• Segmentation-Reconstruction-Guided Facial Image De-occlusion [48.952656891182826]
  Occlusions are very common in face images in the wild, leading to the degraded performance of face-related tasks. This paper proposes a novel face de-occlusion model based on face segmentation and 3D face reconstruction.
  arXiv  Detail & Related papers  (2021-12-15T10:40:08Z)
• MLFW: A Database for Face Recognition on Masked Faces [56.441078419992046]
  Masked LFW (MLFW) is a tool to generate masked faces from unmasked faces automatically. The recognition accuracy of SOTA models declines 5%-16% on MLFW database compared with the accuracy on the original images.
  arXiv  Detail & Related papers  (2021-09-13T09:30:10Z)
• HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping [116.1022638063613]
  We propose HifiFace, which can preserve the face shape of the source face and generate photo-realistic results. We introduce the Semantic Facial Fusion module to optimize the combination of encoder and decoder features.
  arXiv  Detail & Related papers  (2021-06-18T07:39:09Z)
• TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection [53.98866801690342]
  3D mask face presentation attack detection (PAD) plays a vital role in securing face recognition systems from 3D mask attacks. We propose a pure r transformer (TransR) framework for learning live intrinsicness representation efficiently. Our TransR is lightweight and efficient (with only 547K parameters and 763MOPs) which is promising for mobile-level applications.
  arXiv  Detail & Related papers  (2021-04-15T12:33:13Z)
• Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection [103.7264459186552]
  Face presentation attack detection (PAD) is essential to secure face recognition systems. Most existing 3D mask PAD benchmarks suffer from several drawbacks. We introduce a largescale High-Fidelity Mask dataset to bridge the gap to real-world applications.
  arXiv  Detail & Related papers  (2021-04-13T12:48:38Z)
• MaskedFace-Net -- A Dataset of Correctly/Incorrectly Masked Face Images in the Context of COVID-19 [2.7528170226206443]
  The wearing of the face masks appears as a solution for limiting the spread of COVID-19. To perform this task, a large dataset of masked faces is necessary for training deep learning models towards detecting people wearing masks and those not wearing masks. Some large datasets of masked faces are available in the literature. However, at the moment, there are no available large dataset of masked face images that permits to check if detected masked faces are correctly worn or not.
  arXiv  Detail & Related papers  (2020-08-18T16:38:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.