DFME: A New Benchmark for Dynamic Facial Micro-expression Recognition

• URL: http://arxiv.org/abs/2301.00985v2
• Date: Sat, 28 Dec 2024 01:22:26 GMT
• Title: DFME: A New Benchmark for Dynamic Facial Micro-expression Recognition
• Authors: Sirui Zhao, Huaying Tang, Xinglong Mao, Shifeng Liu, Yiming Zhang, Hao Wang, Tong Xu, Enhong Chen,
• Abstract summary: Micro-expression (ME) is a spontaneous, subtle, and transient facial expression that reveals human beings genuine emotion.<n>The ME data scarcity has severely hindered the development of advanced data-driven MER models.<n>In this paper, we overcome the ME data scarcity problem by collecting and annotating a dynamic spontaneous ME database.
• Score: 51.26943074578153
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the most important subconscious reactions, micro-expression (ME), is a spontaneous, subtle, and transient facial expression that reveals human beings' genuine emotion. Therefore, automatically recognizing ME (MER) is becoming increasingly crucial in the field of affective computing, providing essential technical support for lie detection, clinical psychological diagnosis, and public safety. However, the ME data scarcity has severely hindered the development of advanced data-driven MER models. Despite the recent efforts by several spontaneous ME databases to alleviate this problem, there is still a lack of sufficient data. Hence, in this paper, we overcome the ME data scarcity problem by collecting and annotating a dynamic spontaneous ME database with the largest current ME data scale called DFME (Dynamic Facial Micro-expressions). Specifically, the DFME database contains 7,526 well-labeled ME videos spanning multiple high frame rates, elicited by 671 participants and annotated by more than 20 professional annotators over three years. Furthermore, we comprehensively verify the created DFME, including using influential spatiotemporal video feature learning models and MER models as baselines, and conduct emotion classification and ME action unit classification experiments. The experimental results demonstrate that the DFME database can facilitate research in automatic MER, and provide a new benchmark for this field. DFME will be published via https://mea-lab-421.github.io.

Related papers

• Autonomous Droplet Microfluidic Design Framework with Large Language Models [0.6827423171182153]
  This study presents MicroFluidic-LLMs, a framework designed for processing and feature extraction. It overcomes processing challenges by transforming the content into a linguistic format and leveraging pre-trained large language models. We demonstrate that our MicroFluidic-LLMs framework can empower deep neural network models to be highly effective and straightforward.
  arXiv  Detail & Related papers  (2024-11-11T03:20:53Z)
• FoME: A Foundation Model for EEG using Adaptive Temporal-Lateral Attention Scaling [19.85701025524892]
  FoME (Foundation Model for EEG) is a novel approach using adaptive temporal-lateral attention scaling. FoME is pre-trained on a diverse 1.7TB dataset of scalp and intracranial EEG recordings, comprising 745M parameters trained for 1,096k steps.
  arXiv  Detail & Related papers  (2024-09-19T04:22:40Z)
• Knowledge Graph Enhanced Retrieval-Augmented Generation for Failure Mode and Effects Analysis [1.8849131083278732]
  Failure mode and effects analysis (FMEA) is a critical tool for mitigating potential failures. Large language models (LLMs) offer novel prospects for fine-tuning on custom datasets for reasoning within FMEA contexts. Retrieval-augmented generation (RAG) approaches aim to fill this gap.
  arXiv  Detail & Related papers  (2024-06-26T07:02:49Z)
• Towards Precision Healthcare: Robust Fusion of Time Series and Image Data [8.579651833717763]
  We introduce a new method that uses two separate encoders, one for each type of data, allowing the model to understand complex patterns in both visual and time-based information. We also deal with imbalanced datasets and use an uncertainty loss function, yielding improved results. Our experiments show that our method is effective in improving multimodal deep learning for clinical applications.
  arXiv  Detail & Related papers  (2024-05-24T11:18:13Z)
• MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild [81.32127423981426]
  Multimodal emotion recognition based on audio and video data is important for real-world applications. Recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multimodal encoders. We propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders.
  arXiv  Detail & Related papers  (2024-04-13T13:39:26Z)
• InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling [66.3072381478251]
  Reward hacking, also termed reward overoptimization, remains a critical challenge. We propose a framework for reward modeling, namely InfoRM, by introducing a variational information bottleneck objective. We show that InfoRM's overoptimization detection mechanism is not only effective but also robust across a broad range of datasets.
  arXiv  Detail & Related papers  (2024-02-14T17:49:07Z)
• MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition [47.29528724322795]
  MAE-DFER is a novel self-supervised method for learning dynamic facial expressions. It uses large-scale self-supervised pre-training on abundant unlabeled data. It consistently outperforms state-of-the-art supervised methods.
  arXiv  Detail & Related papers  (2023-07-05T12:08:56Z)
• Deeply-Learned Generalized Linear Models with Missing Data [6.302686933168439]
  We provide a formal treatment of missing data in the context of deeply learned generalized linear models. We propose a new architecture, textitdlglm, that is able to flexibly account for both ignorable and non-ignorable patterns of missingness. We conclude with a case study of a Bank Marketing dataset from the UCI Machine Learning Repository.
  arXiv  Detail & Related papers  (2022-07-18T20:00:13Z)
• Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms [52.58031087639394]
  micro-expressions are involuntary and transient facial expressions. They can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, their detection and recognition is difficult and relies heavily on expert experiences.
  arXiv  Detail & Related papers  (2022-01-30T05:14:13Z)
• Deep Learning based Micro-expression Recognition: A Survey [34.14579226321051]
  Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations. With the success of deep learning (DL) in various fields, neural networks have received increasing interests in MER. This survey defines a new taxonomy for the field, encompassing all aspects of MER based on DL.
  arXiv  Detail & Related papers  (2021-07-06T18:05:52Z)
• Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship [63.27052967981546]
  We introduce the first publicly available multi-task MM kinship dataset. To build FIW MM, we developed machinery to automatically collect, annotate, and prepare the data. Results highlight edge cases to inspire future research with different areas of improvement.
  arXiv  Detail & Related papers  (2020-07-28T22:36:57Z)
• Micro-expression spotting: A new benchmark [74.69928316848866]
  Micro-expressions (MEs) are brief and involuntary facial expressions that occur when people are trying to hide their true feelings or conceal their emotions. In the computer vision field, the study of MEs can be divided into two main tasks, spotting and recognition. This paper introduces an extension of the SMIC-E database, namely the SMIC-E-Long database, which is a new challenging benchmark for ME spotting.
  arXiv  Detail & Related papers  (2020-07-24T09:18:41Z)
• Students Need More Attention: BERT-based AttentionModel for Small Data with Application to AutomaticPatient Message Triage [65.7062363323781]
  We propose a novel framework based on BioBERT (Bidirectional Representations from Transformers forBiomedical TextMining) We introduce Label Embeddings for Self-Attention in each layer of BERT, which we call LESA-BERT, and (ii) by distilling LESA-BERT to smaller variants, we aim to reduce overfitting and model size when working on small datasets. As an application, our framework is utilized to build a model for patient portal message triage that classifies the urgency of a message into three categories: non-urgent, medium and urgent.
  arXiv  Detail & Related papers  (2020-06-22T03:39:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.