Real-Time Facial Expression Emoji Masking with Convolutional Neural Networks and Homography

• URL: http://arxiv.org/abs/2012.13447v1
• Date: Thu, 24 Dec 2020 21:25:48 GMT
• Title: Real-Time Facial Expression Emoji Masking with Convolutional Neural Networks and Homography
• Authors: Qinchen Wang and Sixuan Wu and Tingfeng Xia
• Abstract summary: In image processing, Convolutional Neural Networks (CNN) can be trained to categorize facial expressions of images of human faces. In this work, we create a system that masks a student's face with a emoji of the respective emotion. Our results show that this pipeline is deploy-able in real-time, and is usable in educational settings.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Neural network based algorithms has shown success in many applications. In image processing, Convolutional Neural Networks (CNN) can be trained to categorize facial expressions of images of human faces. In this work, we create a system that masks a student's face with a emoji of the respective emotion. Our system consists of three building blocks: face detection using Histogram of Gradients (HoG) and Support Vector Machine (SVM), facial expression categorization using CNN trained on FER2013 dataset, and finally masking the respective emoji back onto the student's face via homography estimation. (Demo: https://youtu.be/GCjtXw1y8Pw) Our results show that this pipeline is deploy-able in real-time, and is usable in educational settings.

Related papers

• Alleviating Catastrophic Forgetting in Facial Expression Recognition with Emotion-Centered Models [49.3179290313959]
  The proposed method, emotion-centered generative replay (ECgr), tackles this challenge by integrating synthetic images from generative adversarial networks. ECgr incorporates a quality assurance algorithm to ensure the fidelity of generated images. The experimental results on four diverse facial expression datasets demonstrate that incorporating images generated by our pseudo-rehearsal method enhances training on the targeted dataset and the source dataset.
  arXiv  Detail & Related papers  (2024-04-18T15:28:34Z)
• GiMeFive: Towards Interpretable Facial Emotion Classification [1.1468563069298348]
  Deep convolutional neural networks have been shown to successfully recognize facial emotions. We propose our model GiMeFive with interpretations, i.e., via layer activations and gradient-weighted class mapping. Empirical results show that our model outperforms the previous methods in terms of accuracy.
  arXiv  Detail & Related papers  (2024-02-24T00:37:37Z)
• Graphics Capsule: Learning Hierarchical 3D Face Representations from 2D Images [82.5266467869448]
  We propose an Inverse Graphics Capsule Network (IGC-Net) to learn the hierarchical 3D face representations from large-scale unlabeled images. IGC-Net first decomposes the objects into a set of semantic-consistent part-level descriptions and then assembles them into object-level descriptions to build the hierarchy.
  arXiv  Detail & Related papers  (2023-03-20T06:32:55Z)
• An Approach for Improving Automatic Mouth Emotion Recognition [1.5293427903448025]
  The study proposes and tests a technique for automated emotion recognition through mouth detection via Convolutional Neural Networks (CNN) The technique is meant to be applied for supporting people with health disorders with communication skills issues.
  arXiv  Detail & Related papers  (2022-12-12T16:17:21Z)
• Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers [57.1091606948826]
  We propose a novel FER model, named Poker Face Vision Transformer or PF-ViT, to address these challenges. PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face. PF-ViT utilizes vanilla Vision Transformers, and its components are pre-trained as Masked Autoencoders on a large facial expression dataset.
  arXiv  Detail & Related papers  (2022-07-22T13:39:06Z)
• Real-Time Facial Expression Recognition using Facial Landmarks and Neural Networks [0.0]
  This paper presents an algorithm for feature extraction, classification of seven different emotions, and facial expression recognition in a real-time manner. A Multi-Layer Perceptron neural network is trained based on the foregoing algorithm. A 3-layer is trained using these feature vectors, leading to 96% accuracy on the test set.
  arXiv  Detail & Related papers  (2022-01-31T21:38:30Z)
• Learning Continuous Face Representation with Explicit Functions [20.5159277443333]
  We propose an explicit model (EmFace) for human face representation in the form of a finite sum of mathematical terms. EmFace achieves reasonable performance on several face image processing tasks, including face image restoration, denoising, and transformation.
  arXiv  Detail & Related papers  (2021-10-25T03:49:20Z)
• Continuous Emotion Recognition with Spatiotemporal Convolutional Neural Networks [82.54695985117783]
  We investigate the suitability of state-of-the-art deep learning architectures for continuous emotion recognition using long video sequences captured in-the-wild. We have developed and evaluated convolutional recurrent neural networks combining 2D-CNNs and long short term-memory units, and inflated 3D-CNN models, which are built by inflating the weights of a pre-trained 2D-CNN model during fine-tuning.
  arXiv  Detail & Related papers  (2020-11-18T13:42:05Z)
• Synthetic Expressions are Better Than Real for Learning to Detect Facial Actions [4.4532095214807965]
  Our approach reconstructs the 3D shape of the face from each video frame, aligns the 3D mesh to a canonical view, and then trains a GAN-based network to synthesize novel images with facial action units of interest. The network trained on synthesized facial expressions outperformed the one trained on actual facial expressions and surpassed current state-of-the-art approaches.
  arXiv  Detail & Related papers  (2020-10-21T13:11:45Z)
• DeepFaceFlow: In-the-wild Dense 3D Facial Motion Estimation [56.56575063461169]
  DeepFaceFlow is a robust, fast, and highly-accurate framework for the estimation of 3D non-rigid facial flow. Our framework was trained and tested on two very large-scale facial video datasets. Given registered pairs of images, our framework generates 3D flow maps at 60 fps.
  arXiv  Detail & Related papers  (2020-05-14T23:56:48Z)
• Exploiting Semantics for Face Image Deblurring [121.44928934662063]
  We propose an effective and efficient face deblurring algorithm by exploiting semantic cues via deep convolutional neural networks. We incorporate face semantic labels as input priors and propose an adaptive structural loss to regularize facial local structures. The proposed method restores sharp images with more accurate facial features and details.
  arXiv  Detail & Related papers  (2020-01-19T13:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.