A PCA based Keypoint Tracking Approach to Automated Facial Expressions Encoding

• URL: http://arxiv.org/abs/2406.09017v1
• Date: Thu, 13 Jun 2024 11:40:26 GMT
• Title: A PCA based Keypoint Tracking Approach to Automated Facial Expressions Encoding
• Authors: Shivansh Chandra Tripathi, Rahul Garg,
• Abstract summary: This paper explores the use of automated techniques to generate Action Units (AUs) for studying facial expressions. We propose an unsupervised approach based on Principal Component Analysis (PCA) and facial keypoint tracking to generate data-driven AUs.
• Score: 3.0605062268685868
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Facial Action Coding System (FACS) for studying facial expressions is manual and requires significant effort and expertise. This paper explores the use of automated techniques to generate Action Units (AUs) for studying facial expressions. We propose an unsupervised approach based on Principal Component Analysis (PCA) and facial keypoint tracking to generate data-driven AUs called PCA AUs using the publicly available DISFA dataset. The PCA AUs comply with the direction of facial muscle movements and are capable of explaining over 92.83 percent of the variance in other public test datasets (BP4D-Spontaneous and CK+), indicating their capability to generalize facial expressions. The PCA AUs are also comparable to a keypoint-based equivalence of FACS AUs in terms of variance explained on the test datasets. In conclusion, our research demonstrates the potential of automated techniques to be an alternative to manual FACS labeling which could lead to efficient real-time analysis of facial expressions in psychology and related fields. To promote further research, we have made code repository publicly available.

Related papers

• Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism [3.0274846041592864]
  The Facial Action Coding System (FACS) has been used by numerous studies to investigate the links between facial behavior and mental health.<n>Despite intense efforts spanning three decades, the detection accuracy for many Action Units is considered to be below the threshold needed for behavioral research.<n>This paper proposes a new coding system that mimics the key properties of FACS.
  arXiv  Detail & Related papers  (2025-05-30T15:06:01Z)
• Representation Learning and Identity Adversarial Training for Facial Behavior Understanding [3.350769246260559]
  We show that subject identity provides a shortcut learning for the model and leads to sub-optimal solutions to AU predictions. We propose Identity Adrial Training (IAT) and demonstrate that a strong IAT regularization is necessary to learn identity-invariant features. Our proposed methods, Facial Masked Autoencoder (FMAE) and IAT, are simple, generic and effective.
  arXiv  Detail & Related papers  (2024-07-15T21:13:28Z)
• Unsupervised learning of Data-driven Facial Expression Coding System (DFECS) using keypoint tracking [3.0605062268685868]
  We propose an unsupervised learning of an automated facial coding system by leveraging computer-vision-based facial keypoint tracking. Results show that DFECS AUs estimated from the DISFA dataset can account for an average variance of up to 91.29 percent in test datasets. 87.5 percent of DFECS AUs are interpretable, i.e., align with the direction of facial muscle movements.
  arXiv  Detail & Related papers  (2024-06-08T10:45:38Z)
• Contrastive Learning of Person-independent Representations for Facial Action Unit Detection [70.60587475492065]
  We formulate the self-supervised AU representation learning signals in two-fold. We contrast learn the AU representation within a video clip and devise a cross-identity reconstruction mechanism to learn the person-independent representations. Our method outperforms other contrastive learning methods and significantly closes the performance gap between the self-supervised and supervised AU detection approaches.
  arXiv  Detail & Related papers  (2024-03-06T01:49:28Z)
• Unsupervised Sentiment Analysis of Plastic Surgery Social Media Posts [91.3755431537592]
  The massive collection of user posts across social media platforms is primarily untapped for artificial intelligence (AI) use cases. Natural language processing (NLP) is a subfield of AI that leverages bodies of documents, known as corpora, to train computers in human-like language understanding. This study demonstrates that the applied results of unsupervised analysis allow a computer to predict either negative, positive, or neutral user sentiment towards plastic surgery.
  arXiv  Detail & Related papers  (2023-07-05T20:16:20Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• AU-Supervised Convolutional Vision Transformers for Synthetic Facial Expression Recognition [12.661683851729679]
  The paper describes our proposed methodology for the six basic expression classification track of Affective Behavior Analysis in-the-wild (ABAW) Competition 2022. Because of the ambiguous of the synthetic data and the objectivity of the facial Action Unit (AU), we resort to the AU information for performance boosting.
  arXiv  Detail & Related papers  (2022-07-20T09:33:39Z)
• Evaluation of Self-taught Learning-based Representations for Facial Emotion Recognition [62.30451764345482]
  This work describes different strategies to generate unsupervised representations obtained through the concept of self-taught learning for facial emotion recognition. The idea is to create complementary representations promoting diversity by varying the autoencoders' initialization, architecture, and training data. Experimental results on Jaffe and Cohn-Kanade datasets using a leave-one-subject-out protocol show that FER methods based on the proposed diverse representations compare favorably against state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-04-26T22:48:15Z)
• Feature Visualization within an Automated Design Assessment leveraging Explainable Artificial Intelligence Methods [0.0]
  Automated capability assessment, mainly leveraged by deep learning systems driven from 3D CAD data, have been presented. Current assessment systems may be able to assess CAD data with regards to abstract features, but without any geometrical indicator about the reasons of the system's decision. Within the NeuroCAD Project, xAI methods are used to identify geometrical features which are associated with a certain abstract feature.
  arXiv  Detail & Related papers  (2022-01-28T13:31:42Z)
• Human-in-the-Loop Disinformation Detection: Stance, Sentiment, or Something Else? [93.91375268580806]
  Both politics and pandemics have recently provided ample motivation for the development of machine learning-enabled disinformation (a.k.a. fake news) detection algorithms. Existing literature has focused primarily on the fully-automated case, but the resulting techniques cannot reliably detect disinformation on the varied topics, sources, and time scales required for military applications. By leveraging an already-available analyst as a human-in-the-loop, canonical machine learning techniques of sentiment analysis, aspect-based sentiment analysis, and stance detection become plausible methods to use for a partially-automated disinformation detection system.
  arXiv  Detail & Related papers  (2021-11-09T13:30:34Z)
• AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition [79.8779790682205]
  We propose an AU-Expression Knowledge Constrained Representation Learning (AUE-CRL) framework to learn the AU representations without AU annotations and adaptively use representations to facilitate facial expression recognition. We conduct experiments on the challenging uncontrolled datasets to demonstrate the superiority of the proposed framework over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-12-29T03:42:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.