Customizable Avatars with Dynamic Facial Action Coded Expressions (CADyFACE) for Improved User Engagement

• URL: http://arxiv.org/abs/2403.07314v1
• Date: Tue, 12 Mar 2024 05:00:38 GMT
• Title: Customizable Avatars with Dynamic Facial Action Coded Expressions (CADyFACE) for Improved User Engagement
• Authors: Megan A. Witherow, Crystal Butler, Winston J. Shields, Furkan Ilgin, Norou Diawara, Janice Keener, John W. Harrington, and Khan M. Iftekharuddin
• Abstract summary: 3D avatar-based facial expression stimuli may improve user engagement in behavioral biomarker discovery. There is a lack of customizable avatar-based stimuli with Facial Action Coding System (FACS) action unit (AU) labels. This study focuses on (1) FACS-labeled, customizable avatar-based expression stimuli for maintaining subjects' engagement, (2) learning-based measurements that quantify subjects' facial responses to such stimuli, and (3) validation of constructs represented by-measurement stimulus pairs.
• Score: 0.5358896402695404
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Customizable 3D avatar-based facial expression stimuli may improve user engagement in behavioral biomarker discovery and therapeutic intervention for autism, Alzheimer's disease, facial palsy, and more. However, there is a lack of customizable avatar-based stimuli with Facial Action Coding System (FACS) action unit (AU) labels. Therefore, this study focuses on (1) FACS-labeled, customizable avatar-based expression stimuli for maintaining subjects' engagement, (2) learning-based measurements that quantify subjects' facial responses to such stimuli, and (3) validation of constructs represented by stimulus-measurement pairs. We propose Customizable Avatars with Dynamic Facial Action Coded Expressions (CADyFACE) labeled with AUs by a certified FACS expert. To measure subjects' AUs in response to CADyFACE, we propose a novel Beta-guided Correlation and Multi-task Expression learning neural network (BeCoME-Net) for multi-label AU detection. The beta-guided correlation loss encourages feature correlation with AUs while discouraging correlation with subject identities for improved generalization. We train BeCoME-Net for unilateral and bilateral AU detection and compare with state-of-the-art approaches. To assess construct validity of CADyFACE and BeCoME-Net, twenty healthy adult volunteers complete expression recognition and mimicry tasks in an online feasibility study while webcam-based eye-tracking and video are collected. We test validity of multiple constructs, including face preference during recognition and AUs during mimicry.

Related papers

• Hierarchical Vision-Language Interaction for Facial Action Unit Detection [44.02409932746335]
  We propose a Hierarchical Vision-language Interaction for AU Understanding (HiVA) method to enhance AU detection.<n>HiVA employs a large language model to generate diverse and contextually rich AU descriptions to strengthen language-based representation learning.<n> experiments show that HiVA consistently surpasses state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-02-16T03:22:05Z)
• Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis [20.372029918328035]
  Facial Emotion Analysis (FEA) extends traditional facial emotion recognition by incorporating explainable, fine-grained reasoning.<n>Recent approaches leverage Vision-Language Models (VLMs) and achieve promising results, but they face two critical limitations.<n>We propose Facial-R1, a three-stage alignment framework that effectively addresses both challenges with minimal supervision.
  arXiv  Detail & Related papers  (2025-11-13T12:40:21Z)
• Beyond FACS: Data-driven Facial Expression Dictionaries, with Application to Predicting Autism [3.0274846041592864]
  The Facial Action Coding System (FACS) has been used by numerous studies to investigate the links between facial behavior and mental health.<n>Despite intense efforts spanning three decades, the detection accuracy for many Action Units is considered to be below the threshold needed for behavioral research.<n>This paper proposes a new coding system that mimics the key properties of FACS.
  arXiv  Detail & Related papers  (2025-05-30T15:06:01Z)
• CLIP Unreasonable Potential in Single-Shot Face Recognition [0.0]
  Face recognition is a core task in computer vision designed to identify and authenticate individuals by analyzing facial patterns and features. Recent Contrastive Language Image Pretraining (CLIP) a model developed by OpenAI has shown promising advancements. CLIP links natural language processing with vision tasks allowing it to generalize across modalities.
  arXiv  Detail & Related papers  (2024-11-19T08:23:52Z)
• Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features [50.82725748981231]
  Engagement measurement finds application in healthcare, education, services. Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors. We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
  arXiv  Detail & Related papers  (2024-04-05T20:39:16Z)
• Contrastive Learning of Person-independent Representations for Facial Action Unit Detection [70.60587475492065]
  We formulate the self-supervised AU representation learning signals in two-fold. We contrast learn the AU representation within a video clip and devise a cross-identity reconstruction mechanism to learn the person-independent representations. Our method outperforms other contrastive learning methods and significantly closes the performance gap between the self-supervised and supervised AU detection approaches.
  arXiv  Detail & Related papers  (2024-03-06T01:49:28Z)
• Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection [70.96299509159981]
  Human-Object Interaction (HOI) detection is a core task for human-centric image understanding. Recent one-stage methods adopt a transformer decoder to collect image-wide cues that are useful for interaction prediction. Traditional two-stage methods benefit significantly from their ability to compose interaction features in a disentangled and explainable manner.
  arXiv  Detail & Related papers  (2023-12-04T08:02:59Z)
• CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition [80.07590100872548]
  We propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improvement in facial expression recognition performance over six different datasets with very unique affective representations.
  arXiv  Detail & Related papers  (2022-08-10T15:46:05Z)
• Towards Privacy-Preserving Affect Recognition: A Two-Level Deep Learning Architecture [2.9392867898439006]
  We propose a two-level deep learning architecture for affect recognition. The architecture consists of recurrent neural networks to capture the temporal relationships amongst the features.
  arXiv  Detail & Related papers  (2021-11-14T13:52:57Z)
• Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition [9.188777864190204]
  We introduce our submission to the 2nd Affective Behavior Analysis in-the-wild (ABAW2) Competition. In dealing with different emotion representations, we propose a multi-task streaming network. We leverage an advanced facial expression embedding as prior knowledge.
  arXiv  Detail & Related papers  (2021-07-08T09:35:08Z)
• AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition [79.8779790682205]
  We propose an AU-Expression Knowledge Constrained Representation Learning (AUE-CRL) framework to learn the AU representations without AU annotations and adaptively use representations to facilitate facial expression recognition. We conduct experiments on the challenging uncontrolled datasets to demonstrate the superiority of the proposed framework over current state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-12-29T03:42:04Z)
• Continuous Emotion Recognition via Deep Convolutional Autoencoder and Support Vector Regressor [70.2226417364135]
  It is crucial that the machine should be able to recognize the emotional state of the user with high accuracy. Deep neural networks have been used with great success in recognizing emotions. We present a new model for continuous emotion recognition based on facial expression recognition.
  arXiv  Detail & Related papers  (2020-01-31T17:47:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.