Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GAN

• URL: http://arxiv.org/abs/2212.09098v5
• Date: Tue, 30 May 2023 17:07:58 GMT
• Title: Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GAN
• Authors: Lei Li, Tianfang Zhang, Zhongfeng Kang, Xikun Jiang
• Abstract summary: Mask-FPAN is a novel framework that learns to parse occluded faces in a semi-supervised way. A 3D morphable face model combined with the UV GAN improves the robustness of 2D face parsing. We introduce two new datasets named FaceOcc-MaskHQ and CelebAMaskOcc-HQ for face paring work.
• Score: 9.433856779172064
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-grained semantic segmentation of a person's face and head, including facial parts and head components, has progressed a great deal in recent years. However, it remains a challenging task, whereby considering ambiguous occlusions and large pose variations are particularly difficult. To overcome these difficulties, we propose a novel framework termed Mask-FPAN. It uses a de-occlusion module that learns to parse occluded faces in a semi-supervised way. In particular, face landmark localization, face occlusionstimations, and detected head poses are taken into account. A 3D morphable face model combined with the UV GAN improves the robustness of 2D face parsing. In addition, we introduce two new datasets named FaceOccMask-HQ and CelebAMaskOcc-HQ for face paring work. The proposed Mask-FPAN framework addresses the face parsing problem in the wild and shows significant performance improvements with MIOU from 0.7353 to 0.9013 compared to the state-of-the-art on challenging face datasets.

Related papers

• FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders [81.21440457805932]
  We propose a novel framework FaceMAE, where the face privacy and recognition performance are considered simultaneously. randomly masked face images are used to train the reconstruction module in FaceMAE. We also perform sufficient privacy-preserving face recognition on several public face datasets.
  arXiv  Detail & Related papers  (2022-05-23T07:19:42Z)
• FaceOcc: A Diverse, High-quality Face Occlusion Dataset for Human Face Extraction [3.8502825594372703]
  Occlusions often occur in face images in the wild, troubling face-related tasks such as landmark detection, 3D reconstruction, and face recognition. This paper proposes a novel face segmentation dataset with manually labeled face occlusions from the CelebA-HQ and the internet. We trained a straightforward face segmentation model but obtained SOTA performance, convincingly demonstrating the effectiveness of the proposed dataset.
  arXiv  Detail & Related papers  (2022-01-20T19:44:18Z)
• Segmentation-Reconstruction-Guided Facial Image De-occlusion [48.952656891182826]
  Occlusions are very common in face images in the wild, leading to the degraded performance of face-related tasks. This paper proposes a novel face de-occlusion model based on face segmentation and 3D face reconstruction.
  arXiv  Detail & Related papers  (2021-12-15T10:40:08Z)
• End2End Occluded Face Recognition by Masking Corrupted Features [82.27588990277192]
  State-of-the-art general face recognition models do not generalize well to occluded face images. This paper presents a novel face recognition method that is robust to occlusions based on a single end-to-end deep neural network. Our approach, named FROM (Face Recognition with Occlusion Masks), learns to discover the corrupted features from the deep convolutional neural networks, and clean them by the dynamically learned masks.
  arXiv  Detail & Related papers  (2021-08-21T09:08:41Z)
• Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection [65.92058628082322]
  Non-parametric face modeling aims to reconstruct 3D face only from images without shape assumptions. This paper presents a novel Learning to Aggregate and Personalize framework for unsupervised robust 3D face modeling.
  arXiv  Detail & Related papers  (2021-06-15T03:10:17Z)
• Joint Face Image Restoration and Frontalization for Recognition [79.78729632975744]
  In real-world scenarios, many factors may harm face recognition performance, e.g., large pose, bad illumination,low resolution, blur and noise. Previous efforts usually first restore the low-quality faces to high-quality ones and then perform face recognition. We propose an Multi-Degradation Face Restoration model to restore frontalized high-quality faces from the given low-quality ones.
  arXiv  Detail & Related papers  (2021-05-12T03:52:41Z)
• Towards NIR-VIS Masked Face Recognition [47.00916333095693]
  Near-infrared to visible (NIR-VIS) face recognition is the most common case in heterogeneous face recognition. We propose a novel training method to maximize the mutual information shared by the face representation of two domains. In addition, a 3D face reconstruction based approach is employed to synthesize masked face from the existing NIR image.
  arXiv  Detail & Related papers  (2021-04-14T10:40:09Z)
• Unmasking Face Embeddings by Self-restrained Triplet Loss for Accurate Masked Face Recognition [6.865656740940772]
  We present a solution to improve the masked face recognition performance. Specifically, we propose the Embedding Unmasking Model (EUM) operated on top of existing face recognition models. We also propose a novel loss function, the Self-restrained Triplet (SRT), which enabled the EUM to produce embeddings similar to these of unmasked faces of the same identities.
  arXiv  Detail & Related papers  (2021-03-02T13:43:11Z)
• A 3D model-based approach for fitting masks to faces in the wild [9.958467179573235]
  We present a 3D model-based approach called WearMask3D for augmenting face images of various poses to the masked face counterparts. Our method proceeds by first fitting a 3D morphable model on the input image, second overlaying the mask surface onto the face model and warping the respective mask texture, and last projecting the 3D mask back to 2D. Experimental results demonstrate WearMask3D produces more realistic masked images, and utilizing these images for training leads to improved recognition accuracy of masked faces.
  arXiv  Detail & Related papers  (2021-03-01T06:50:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.