Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective

• URL: http://arxiv.org/abs/2302.10021v2
• Date: Thu, 25 May 2023 19:53:08 GMT
• Title: Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective
• Authors: Nikolaos Kegkeroglou, Panagiotis P. Filntisis, Petros Maragos
• Abstract summary: The COVID-19 pandemic has forced people to extensively wear medical face masks, in order to prevent transmission. This paper conducts insightful studies about the effect of face occlusion on emotion recognition performance. We utilize a deep learning model based on the Temporal Segment Network framework, and aspire to fully overcome the face mask consequences.
• Score: 31.55798962786664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The COVID-19 pandemic has undoubtedly changed the standards and affected all aspects of our lives, especially social communication. It has forced people to extensively wear medical face masks, in order to prevent transmission. This face occlusion can strongly irritate emotional reading from the face and urges us to incorporate the whole body as an emotional cue. In this paper, we conduct insightful studies about the effect of face occlusion on emotion recognition performance, and showcase the superiority of full body input over the plain masked face. We utilize a deep learning model based on the Temporal Segment Network framework, and aspire to fully overcome the face mask consequences. Although facial and bodily features can be learned from a single input, this may lead to irrelevant information confusion. By processing those features separately and fusing their prediction scores, we are more effectively taking advantage of both modalities. This framework also naturally supports temporal modeling, by mingling information among neighboring frames. In combination, these techniques form an effective system capable of tackling emotion recognition difficulties, caused by safety protocols applied in crucial areas.

Related papers

• Seeing through the Mask: Multi-task Generative Mask Decoupling Face Recognition [47.248075664420874]
  Current general face recognition system suffers from serious performance degradation when encountering occluded scenes. This paper proposes a Multi-task gEnerative mask dEcoupling face Recognition (MEER) network to jointly handle these two tasks. We first present a novel mask decoupling module to disentangle mask and identity information, which makes the network obtain purer identity features from visible facial components.
  arXiv  Detail & Related papers  (2023-11-20T03:23:03Z)
• Learning Representations for Masked Facial Recovery [8.124282476398843]
  pandemic of these recent years has led to a dramatic increase in people wearing protective masks in public venues. One way to address the problem is to revert to face recovery methods as a preprocessing step. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask.
  arXiv  Detail & Related papers  (2022-12-28T22:22:15Z)
• PERI: Part Aware Emotion Recognition In The Wild [4.206175795966693]
  This paper focuses on emotion recognition using visual features. We create part aware spatial (PAS) images by extracting key regions from the input image using a mask generated from both body pose and facial landmarks. We provide our results on the publicly available in the wild EMOTIC dataset.
  arXiv  Detail & Related papers  (2022-10-18T20:01:40Z)
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers [57.1091606948826]
  We propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face. PF-ViT utilizes vanilla Vision Transformers, and its components are pre-trained as Masked Autoencoders on a large facial expression dataset.
  arXiv  Detail & Related papers  (2022-07-22T13:39:06Z)
• Adversarial Mask: Real-World Adversarial Attack Against Face Recognition Models [66.07662074148142]
  We propose a physical adversarial universal perturbation (UAP) against state-of-the-art deep learning-based facial recognition models. In our experiments, we examined the transferability of our adversarial mask to a wide range of deep learning models and datasets. We validated our adversarial mask effectiveness in real-world experiments by printing the adversarial pattern on a fabric medical face mask.
  arXiv  Detail & Related papers  (2021-11-21T08:13:21Z)
• I Only Have Eyes for You: The Impact of Masks On Convolutional-Based Facial Expression Recognition [78.07239208222599]
  We evaluate how the recently proposed FaceChannel adapts towards recognizing facial expressions from persons with masks. We also perform specific feature-level visualization to demonstrate how the inherent capabilities of the FaceChannel to learn and combine facial features change when in a constrained social interaction scenario.
  arXiv  Detail & Related papers  (2021-04-16T20:03:30Z)
• Facial Expressions as a Vulnerability in Face Recognition [73.85525896663371]
  This work explores facial expression bias as a security vulnerability of face recognition systems. We present a comprehensive analysis of how facial expression bias impacts the performance of face recognition technologies.
  arXiv  Detail & Related papers  (2020-11-17T18:12:41Z)
• Learning Emotional-Blinded Face Representations [77.7653702071127]
  We propose two face representations that are blind to facial expressions associated to emotional responses. This work is motivated by new international regulations for personal data protection.
  arXiv  Detail & Related papers  (2020-09-18T09:24:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.