iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding
and Emotion Analysis
- URL: http://arxiv.org/abs/2107.00285v1
- Date: Thu, 1 Jul 2021 08:15:14 GMT
- Title: iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding
and Emotion Analysis
- Authors: Xin Liu, Henglin Shi, Haoyu Chen, Zitong Yu, Xiaobai Li, Guoying
Zhaoz?
- Abstract summary: iMiGUE is identity-free video dataset for Micro-Gesture Understanding and Emotion analysis (iMiGUE)
iMiGUE focuses on micro-gesture, i.e., unintentional behaviors driven by inner feelings.
- Score: 23.261770969903065
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We introduce a new dataset for the emotional artificial intelligence
research: identity-free video dataset for Micro-Gesture Understanding and
Emotion analysis (iMiGUE). Different from existing public datasets, iMiGUE
focuses on nonverbal body gestures without using any identity information,
while the predominant researches of emotion analysis concern sensitive
biometric data, like face and speech. Most importantly, iMiGUE focuses on
micro-gestures, i.e., unintentional behaviors driven by inner feelings, which
are different from ordinary scope of gestures from other gesture datasets which
are mostly intentionally performed for illustrative purposes. Furthermore,
iMiGUE is designed to evaluate the ability of models to analyze the emotional
states by integrating information of recognized micro-gesture, rather than just
recognizing prototypes in the sequences separately (or isolatedly). This is
because the real need for emotion AI is to understand the emotional states
behind gestures in a holistic way. Moreover, to counter for the challenge of
imbalanced sample distribution of this dataset, an unsupervised learning method
is proposed to capture latent representations from the micro-gesture sequences
themselves. We systematically investigate representative methods on this
dataset, and comprehensive experimental results reveal several interesting
insights from the iMiGUE, e.g., micro-gesture-based analysis can promote
emotion understanding. We confirm that the new iMiGUE dataset could advance
studies of micro-gesture and emotion AI.
Related papers
- Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding [21.94739567923136]
We focus on a special group of human body language -- the micro-gesture (MG)
MG differs from the range of ordinary illustrative gestures in that they are not intentional behaviors performed to convey information to others, but rather unintentional behaviors driven by inner feelings.
We explore various augmentation strategies that take into account the subtle spatial and brief temporal characteristics of micro-gestures, often accompanied by repetitiveness, to determine more suitable augmentation methods.
arXiv Detail & Related papers (2024-05-21T21:16:55Z) - Benchmarking Micro-action Recognition: Dataset, Methods, and Applications [26.090557725760934]
Micro-action is imperceptible non-verbal behaviour characterised by low-intensity movement.
In this study, we innovatively collect a new micro-action dataset designated as Micro-action-52 (MA-52)
Uniquely, MA-52 provides the whole-body perspective including gestures, upper- and lower-limb movements.
arXiv Detail & Related papers (2024-03-08T11:48:44Z) - Seeking Subjectivity in Visual Emotion Distribution Learning [93.96205258496697]
Visual Emotion Analysis (VEA) aims to predict people's emotions towards different visual stimuli.
Existing methods often predict visual emotion distribution in a unified network, neglecting the inherent subjectivity in its crowd voting process.
We propose a novel textitSubjectivity Appraise-and-Match Network (SAMNet) to investigate the subjectivity in visual emotion distribution.
arXiv Detail & Related papers (2022-07-25T02:20:03Z) - Video-based Facial Micro-Expression Analysis: A Survey of Datasets,
Features and Algorithms [52.58031087639394]
micro-expressions are involuntary and transient facial expressions.
They can provide important information in a broad range of applications such as lie detection, criminal detection, etc.
Since micro-expressions are transient and of low intensity, their detection and recognition is difficult and relies heavily on expert experiences.
arXiv Detail & Related papers (2022-01-30T05:14:13Z) - SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network [83.27291945217424]
We propose a novel Scene-Object interreLated Visual Emotion Reasoning network (SOLVER) to predict emotions from images.
To mine the emotional relationships between distinct objects, we first build up an Emotion Graph based on semantic concepts and visual features.
We also design a Scene-Object Fusion Module to integrate scenes and objects, which exploits scene features to guide the fusion process of object features with the proposed scene-based attention mechanism.
arXiv Detail & Related papers (2021-10-24T02:41:41Z) - INVIGORATE: Interactive Visual Grounding and Grasping in Clutter [56.00554240240515]
INVIGORATE is a robot system that interacts with human through natural language and grasps a specified object in clutter.
We train separate neural networks for object detection, for visual grounding, for question generation, and for OBR detection and grasping.
We build a partially observable Markov decision process (POMDP) that integrates the learned neural network modules.
arXiv Detail & Related papers (2021-08-25T07:35:21Z) - Analyzing the Influence of Dataset Composition for Emotion Recognition [0.0]
We analyze the influence data collection methodology has on two multimodal emotion recognition datasets.
Experiments with the full IEMOCAP dataset indicate that the composition negatively influences generalization performance when compared to the OMG-Emotion Behavior dataset.
arXiv Detail & Related papers (2021-03-05T14:20:59Z) - A Variational Information Bottleneck Approach to Multi-Omics Data
Integration [98.6475134630792]
We propose a deep variational information bottleneck (IB) approach for incomplete multi-view observations.
Our method applies the IB framework on marginal and joint representations of the observed views to focus on intra-view and inter-view interactions that are relevant for the target.
Experiments on real-world datasets show that our method consistently achieves gain from data integration and outperforms state-of-the-art benchmarks.
arXiv Detail & Related papers (2021-02-05T06:05:39Z) - Latent Feature Representation via Unsupervised Learning for Pattern
Discovery in Massive Electron Microscopy Image Volumes [4.278591555984395]
In particular, we give an unsupervised deep learning approach to learning a latent representation that captures semantic similarity in the data set.
We demonstrate the utility of our method applied to nano-scale electron microscopy data, where even relatively small portions of animal brains can require terabytes of image data.
arXiv Detail & Related papers (2020-12-22T17:14:19Z) - Micro-expression spotting: A new benchmark [74.69928316848866]
Micro-expressions (MEs) are brief and involuntary facial expressions that occur when people are trying to hide their true feelings or conceal their emotions.
In the computer vision field, the study of MEs can be divided into two main tasks, spotting and recognition.
This paper introduces an extension of the SMIC-E database, namely the SMIC-E-Long database, which is a new challenging benchmark for ME spotting.
arXiv Detail & Related papers (2020-07-24T09:18:41Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.