High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network

• URL: http://arxiv.org/abs/2112.11713v1
• Date: Wed, 22 Dec 2021 07:46:23 GMT
• Title: High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network
• Authors: Meng-Tzu Chiu, Hsun-Ying Cheng, Chien-Yi Wang, Shang-Hong Lai
• Abstract summary: Deep learning approaches have achieved highly accurate face recognition by training the models with very large face image datasets. Unlike the availability of large 2D face image datasets, there is a lack of large 3D face datasets available to the public. This paper proposes two CNN models to improve the RGB-D face recognition task.
• Score: 16.50097148165777
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning approaches have achieved highly accurate face recognition by training the models with very large face image datasets. Unlike the availability of large 2D face image datasets, there is a lack of large 3D face datasets available to the public. Existing public 3D face datasets were usually collected with few subjects, leading to the over-fitting problem. This paper proposes two CNN models to improve the RGB-D face recognition task. The first is a segmentation-aware depth estimation network, called DepthNet, which estimates depth maps from RGB face images by including semantic segmentation information for more accurate face region localization. The other is a novel mask-guided RGB-D face recognition model that contains an RGB recognition branch, a depth map recognition branch, and an auxiliary segmentation mask branch with a spatial attention module. Our DepthNet is used to augment a large 2D face image dataset to a large RGB-D face dataset, which is used for training an accurate RGB-D face recognition model. Furthermore, the proposed mask-guided RGB-D face recognition model can fully exploit the depth map and segmentation mask information and is more robust against pose variation than previous methods. Our experimental results show that DepthNet can produce more reliable depth maps from face images with the segmentation mask. Our mask-guided face recognition model outperforms state-of-the-art methods on several public 3D face datasets.

Related papers

• Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Images [11.100398985633754]
  We propose an end-to-end framework for recovering dense meshes for both hands. Our framework employs ResNet50 and PointNet++ to derive features from RGB and point cloud. We also introduce a novel pyramid deep fusion network (PDFNet) to aggregate features at different scales.
  arXiv  Detail & Related papers  (2023-07-12T09:33:21Z)
• Improving 2D face recognition via fine-level facial depth generation and RGB-D complementary feature learning [0.8223798883838329]
  We propose a fine-grained facial depth generation network and an improved multimodal complementary feature learning network. Experiments on the Lock3DFace dataset and the IIIT-D dataset show that the proposed FFDGNet and I MCFLNet can improve the accuracy of RGB-D face recognition.
  arXiv  Detail & Related papers  (2023-05-08T02:33:59Z)
• Generating 2D and 3D Master Faces for Dictionary Attacks with a Network-Assisted Latent Space Evolution [68.8204255655161]
  A master face is a face image that passes face-based identity authentication for a high percentage of the population. We optimize these faces for 2D and 3D face verification models. In 3D, we generate faces using the 2D StyleGAN2 generator and predict a 3D structure using a deep 3D face reconstruction network.
  arXiv  Detail & Related papers  (2022-11-25T09:15:38Z)
• Layered Depth Refinement with Mask Guidance [61.10654666344419]
  We formulate a novel problem of mask-guided depth refinement that utilizes a generic mask to refine the depth prediction of SIDE models. Our framework performs layered refinement and inpainting/outpainting, decomposing the depth map into two separate layers signified by the mask and the inverse mask. We empirically show that our method is robust to different types of masks and initial depth predictions, accurately refining depth values in inner and outer mask boundary regions.
  arXiv  Detail & Related papers  (2022-06-07T06:42:44Z)
• Total Scale: Face-to-Body Detail Reconstruction from Sparse RGBD Sensors [52.38220261632204]
  Flat facial surfaces frequently occur in the PIFu-based reconstruction results. We propose a two-scale PIFu representation to enhance the quality of the reconstructed facial details. Experiments demonstrate the effectiveness of our approach in vivid facial details and deforming body shapes.
  arXiv  Detail & Related papers  (2021-12-03T18:46:49Z)
• Facial Depth and Normal Estimation using Single Dual-Pixel Camera [81.02680586859105]
  We introduce a DP-oriented Depth/Normal network that reconstructs the 3D facial geometry. It contains the corresponding ground-truth 3D models including depth map and surface normal in metric scale. It achieves state-of-the-art performances over recent DP-based depth/normal estimation methods.
  arXiv  Detail & Related papers  (2021-11-25T05:59:27Z)
• Depth as Attention for Face Representation Learning [11.885178256393893]
  A novel depth-guided attention mechanism is proposed for deep multi-modal face recognition using low-cost RGB-D sensors. Our solution achieves average (increased) accuracies of 87.3% (+5.0%), 99.1% (+0.9%), 99.7% (+0.6%) and 95.3%(+0.5%) for the four datasets respectively.
  arXiv  Detail & Related papers  (2021-01-03T16:19:34Z)
• Reconstructing A Large Scale 3D Face Dataset for Deep 3D Face Identification [9.159921061636695]
  We propose a framework of 2D-aided deep 3D face identification. In particular, we propose to reconstruct millions of 3D face scans from a large scale 2D face database. Our proposed approach achieves state-of-the-art rank-1 scores on the FRGC v2.0, Bosphorus, and BU-3DFE 3D face databases.
  arXiv  Detail & Related papers  (2020-10-16T13:48:38Z)
• 3D Dense Geometry-Guided Facial Expression Synthesis by Adversarial Learning [54.24887282693925]
  We propose a novel framework to exploit 3D dense (depth and surface normals) information for expression manipulation. We use an off-the-shelf state-of-the-art 3D reconstruction model to estimate the depth and create a large-scale RGB-Depth dataset. Our experiments demonstrate that the proposed method outperforms the competitive baseline and existing arts by a large margin.
  arXiv  Detail & Related papers  (2020-09-30T17:12:35Z)
• Multi-channel Deep 3D Face Recognition [4.726009758066045]
  The accuracy of 2D face recognition is still challenged by the change of pose, illumination, make-up, and expression. We propose a multi-Channel deep 3D face network for face recognition based on 3D face data. The face recognition accuracy of the multi-Channel deep 3D face network has achieved 98.6.
  arXiv  Detail & Related papers  (2020-09-30T15:29:05Z)
• Face Super-Resolution Guided by 3D Facial Priors [92.23902886737832]
  We propose a novel face super-resolution method that explicitly incorporates 3D facial priors which grasp the sharp facial structures. Our work is the first to explore 3D morphable knowledge based on the fusion of parametric descriptions of face attributes. The proposed 3D priors achieve superior face super-resolution results over the state-of-the-arts.
  arXiv  Detail & Related papers  (2020-07-18T15:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.