Action Units That Constitute Trainable Micro-expressions (and A Large-scale Synthetic Dataset)

• URL: http://arxiv.org/abs/2112.01730v1
• Date: Fri, 3 Dec 2021 06:09:06 GMT
• Title: Action Units That Constitute Trainable Micro-expressions (and A Large-scale Synthetic Dataset)
• Authors: Yuchi Liu, Zhongdao Wang, Tom Gedeon, Liang Zheng
• Abstract summary: We aim to develop a protocol to automatically synthesize micro-expression training data on a large scale. Specifically, we discover three types of Action Units (AUs) that can well constitute trainable micro-expressions. With these AUs, our protocol employs large numbers of face images with various identities and an existing face generation method for micro-expression synthesis. Micro-expression recognition models are trained on the generated micro-expression datasets and evaluated on real-world test sets.
• Score: 20.866448615388876
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the expensive data collection process, micro-expression datasets are generally much smaller in scale than those in other computer vision fields, rendering large-scale training less stable and feasible. In this paper, we aim to develop a protocol to automatically synthesize micro-expression training data that 1) are on a large scale and 2) allow us to train recognition models with strong accuracy on real-world test sets. Specifically, we discover three types of Action Units (AUs) that can well constitute trainable micro-expressions. These AUs come from real-world micro-expressions, early frames of macro-expressions, and the relationship between AUs and expression labels defined by human knowledge. With these AUs, our protocol then employs large numbers of face images with various identities and an existing face generation method for micro-expression synthesis. Micro-expression recognition models are trained on the generated micro-expression datasets and evaluated on real-world test sets, where very competitive and stable performance is obtained. The experimental results not only validate the effectiveness of these AUs and our dataset synthesis protocol but also reveal some critical properties of micro-expressions: they generalize across faces, are close to early-stage macro-expressions, and can be manually defined.

Related papers

• ExpLLM: Towards Chain of Thought for Facial Expression Recognition [61.49849866937758]
  We propose a novel method called ExpLLM to generate an accurate chain of thought (CoT) for facial expression recognition. Specifically, we have designed the CoT mechanism from three key perspectives: key observations, overall emotional interpretation, and conclusion. In experiments on the RAF-DB and AffectNet datasets, ExpLLM outperforms current state-of-the-art FER methods.
  arXiv  Detail & Related papers  (2024-09-04T15:50:16Z)
• Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition [48.21696443824074]
  We propose a novel framework for micro-expression recognition, named the Adaptive Temporal Motion Guided Graph Convolution Network (ATM-GCN) Our framework excels at capturing temporal dependencies between frames across the entire clip, thereby enhancing micro-expression recognition at the clip level.
  arXiv  Detail & Related papers  (2024-06-13T10:57:24Z)
• Efficient Neural Architecture Search for Emotion Recognition [10.944807967751277]
  We propose to search for a highly efficient and robust neural architecture for both macro and micro-expression recognition. We produce lightweight models with a gradient-based architecture search algorithm. The proposed models outperform the existing state-of-the-art methods and perform very well in terms of speed and space complexity.
  arXiv  Detail & Related papers  (2023-03-23T20:21:26Z)
• Data Leakage and Evaluation Issues in Micro-Expression Analysis [45.215233522470115]
  We show that data leakage and fragmented evaluation protocols are issues among the micro-expression literature. We propose a new standardized evaluation protocol using facial action units with over 2000 micro-expression samples, and provide an open source library that implements the evaluation protocols in a standardized manner.
  arXiv  Detail & Related papers  (2022-11-21T13:12:07Z)
• Micro-Expression Recognition Based on Attribute Information Embedding and Cross-modal Contrastive Learning [22.525295392858293]
  We propose a micro-expression recognition method based on attribute information embedding and cross-modal contrastive learning. We conduct extensive experiments in CASME II and MMEW databases, and the accuracy is 77.82% and 71.04%, respectively.
  arXiv  Detail & Related papers  (2022-05-29T12:28:10Z)
• Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms [52.58031087639394]
  micro-expressions are involuntary and transient facial expressions. They can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, their detection and recognition is difficult and relies heavily on expert experiences.
  arXiv  Detail & Related papers  (2022-01-30T05:14:13Z)
• Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition [61.374467942519374]
  We propose a novel a-temporal transformer architecture -- to the best of our knowledge, the first purely transformer based approach for micro-expression recognition. The architecture comprises a spatial encoder which learns spatial patterns, a temporal dimension classification for temporal analysis, and a head. A comprehensive evaluation on three widely used spontaneous micro-expression data sets, shows that the proposed approach consistently outperforms the state of the art.
  arXiv  Detail & Related papers  (2021-12-10T22:10:31Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• ICE-GAN: Identity-aware and Capsule-Enhanced GAN with Graph-based Reasoning for Micro-Expression Recognition and Synthesis [26.414187427071063]
  We propose a novel Identity-aware and Capsule-Enhanced Generative Adversarial Network with graph-based reasoning (ICE-GAN) The generator produces synthetic faces with controllable micro-expressions and identity-aware features, whose long-ranged dependencies are captured through the graph reasoning module (GRM) Our ICE-GAN was evaluated on Micro-Expression Grand Challenge 2019 (MEGC 2019) with a significant improvement (12.9%) over the winner and surpassed other state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-05-09T05:37:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.