Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction

• URL: http://arxiv.org/abs/2012.13217v1
• Date: Thu, 24 Dec 2020 12:28:47 GMT
• Title: Dynamic Facial Expression Recognition under Partial Occlusion with Optical Flow Reconstruction
• Authors: Delphine Poux, Benjamin Allaert, Nacim Ihaddadene, Ioan Marius Bilasco, Chaabane Djeraba and Mohammed Bennamoun
• Abstract summary: We propose a new solution based on an auto-encoder with skip connections to reconstruct the occluded part of the face in the optical flow domain. Our experiments show that the proposed method reduce significantly the gap, in terms of recognition accuracy, between occluded and non-occluded situations.
• Score: 20.28462460359439
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video facial expression recognition is useful for many applications and received much interest lately. Although some solutions give really good results in a controlled environment (no occlusion), recognition in the presence of partial facial occlusion remains a challenging task. To handle occlusions, solutions based on the reconstruction of the occluded part of the face have been proposed. These solutions are mainly based on the texture or the geometry of the face. However, the similarity of the face movement between different persons doing the same expression seems to be a real asset for the reconstruction. In this paper we exploit this asset and propose a new solution based on an auto-encoder with skip connections to reconstruct the occluded part of the face in the optical flow domain. To the best of our knowledge, this is the first proposition to directly reconstruct the movement for facial expression recognition. We validated our approach in the controlled dataset CK+ on which different occlusions were generated. Our experiments show that the proposed method reduce significantly the gap, in terms of recognition accuracy, between occluded and non-occluded situations. We also compare our approach with existing state-of-the-art solutions. In order to lay the basis of a reproducible and fair comparison in the future, we also propose a new experimental protocol that includes occlusion generation and reconstruction evaluation.

Related papers

• CLR-Face: Conditional Latent Refinement for Blind Face Restoration Using Score-Based Diffusion Models [57.9771859175664]
  Recent generative-prior-based methods have shown promising blind face restoration performance. Generating fine-grained facial details faithful to inputs remains a challenging problem. We introduce a diffusion-based-prior inside a VQGAN architecture that focuses on learning the distribution over uncorrupted latent embeddings.
  arXiv  Detail & Related papers  (2024-02-08T23:51:49Z)
• Counterfactual Cross-modality Reasoning for Weakly Supervised Video Moment Localization [67.88493779080882]
  Video moment localization aims to retrieve the target segment of an untrimmed video according to the natural language query. Recent works contrast the cross-modality similarities driven by reconstructing masked queries. We propose a novel proposed counterfactual cross-modality reasoning method.
  arXiv  Detail & Related papers  (2023-08-10T15:45:45Z)
• Latent-OFER: Detect, Mask, and Reconstruct with Latent Vectors for Occluded Facial Expression Recognition [0.0]
  The proposed method can detect occluded parts of the face as if they were unoccluded, and recognize them, improving FER accuracy. It involves three steps: First, the vision transformer (ViT)-based occlusion patch detector masks the occluded position by training only latent vectors from the unoccluded patches. Second, the hybrid reconstruction network generates the masking position as a complete image using the ViT and convolutional neural network (CNN) Last, the expression-relevant latent vector extractor retrieves and uses expression-related information from all latent vectors by applying a CNN-based class activation map
  arXiv  Detail & Related papers  (2023-07-21T07:56:32Z)
• Occlusion Fields: An Implicit Representation for Non-Line-of-Sight Surface Reconstruction [3.0553868534759725]
  Non-line-of-sight reconstruction (NLoS) aims to recover objects outside the field of view from measurements of light that is indirectly scattered off a directly visible, diffuse wall. We propose a new representation and reconstruction technique for NLoS scenes that unifies the treatment of recoverability with the reconstruction itself.
  arXiv  Detail & Related papers  (2022-03-16T14:47:45Z)
• Locality-aware Channel-wise Dropout for Occluded Face Recognition [116.2355331029041]
  Face recognition is a challenging task in unconstrained scenarios, especially when faces are partially occluded. We propose a novel and elegant occlusion-simulation method via dropping the activations of a group of neurons in some elaborately selected channel. Experiments on various benchmarks show that the proposed method outperforms state-of-the-art methods with a remarkable improvement.
  arXiv  Detail & Related papers  (2021-07-20T05:53:14Z)
• Black-Box Face Recovery from Identity Features [61.950765357647605]
  We attack the state-of-the-art face recognition system (ArcFace) to test our algorithm. Our algorithm requires a significantly less number of queries compared to the state-of-the-art solution.
  arXiv  Detail & Related papers  (2020-07-27T15:25:38Z)
• On Improving the Generalization of Face Recognition in the Presence of Occlusions [13.299431908881425]
  Occlusion-aware face REcOgnition (OREO) approach learned discriminative facial templates despite the presence of such occlusions. OREO improved the generalization ability of face recognition under occlusions by (10.17%) in a single-image-based setting.
  arXiv  Detail & Related papers  (2020-06-11T20:17:23Z)
• Occlusion-Adaptive Deep Network for Robust Facial Expression Recognition [56.11054589916299]
  We propose a landmark-guided attention branch to find and discard corrupted features from occluded regions. An attention map is first generated to indicate if a specific facial part is occluded and guide our model to attend to non-occluded regions. This results in more diverse and discriminative features, enabling the expression recognition system to recover even though the face is partially occluded.
  arXiv  Detail & Related papers  (2020-05-12T20:42:55Z)
• Deep Face Super-Resolution with Iterative Collaboration between Attentive Recovery and Landmark Estimation [92.86123832948809]
  We propose a deep face super-resolution (FSR) method with iterative collaboration between two recurrent networks. In each recurrent step, the recovery branch utilizes the prior knowledge of landmarks to yield higher-quality images. A new attentive fusion module is designed to strengthen the guidance of landmark maps.
  arXiv  Detail & Related papers  (2020-03-29T16:04:48Z)
• SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion [7.284661356980246]
  We propose a generative model to reconstruct the missing parts of the face which are under occlusion. A novel adversarial training algorithm has been designed for a bimodal mutually exclusive Generative Adversarial Network (GAN) model. Our proposed technique outperforms the competing methods by a considerable margin, even for boosting the performance of Face Recognition.
  arXiv  Detail & Related papers  (2020-02-19T21:12:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.