Geometric Graph Representation with Learnable Graph Structure and
Adaptive AU Constraint for Micro-Expression Recognition
- URL: http://arxiv.org/abs/2205.00380v1
- Date: Sun, 1 May 2022 02:20:43 GMT
- Title: Geometric Graph Representation with Learnable Graph Structure and
Adaptive AU Constraint for Micro-Expression Recognition
- Authors: Jinsheng Wei and Wei Peng and Guanming Lu and Yante Li and Jingjie Yan
and Guoying Zhao
- Abstract summary: Micro-expression recognition (MER) is valuable because the involuntary nature of micro-expressions (MEs) can reveal genuine emotions.
This paper explores the contribution of facial landmarks and constructs a new framework to efficiently recognize MEs with sole facial landmark information.
The experimental results demonstrate that the proposed method can achieve competitive or even superior performance with a significantly reduced computational cost.
- Score: 38.579316014796945
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Micro-expression recognition (MER) is valuable because the involuntary nature
of micro-expressions (MEs) can reveal genuine emotions. Most works recognize
MEs by taking RGB videos or images as input. In fact, the activated facial
regions in ME images are very small and the subtle motion can be easily
submerged in the unrelated information. Facial landmarks are a low-dimensional
and compact modality, which leads to much lower computational cost and can
potentially concentrate more on ME-related features. However, the
discriminability of landmarks for MER is not clear. Thus, this paper explores
the contribution of facial landmarks and constructs a new framework to
efficiently recognize MEs with sole facial landmark information. Specially, we
design a separate structure module to separately aggregate the spatial and
temporal information in the geometric movement graph based on facial landmarks,
and a Geometric Two-Stream Graph Network is constructed to aggregate the
low-order geometric information and high-order semantic information of facial
landmarks. Furthermore, two core components are proposed to enhance features.
Specifically, a semantic adjacency matrix can automatically model the
relationship between nodes even long-distance nodes in a self-learning fashion;
and an Adaptive Action Unit loss is introduced to guide the learning process
such that the learned features are forced to have a synchronized pattern with
facial action units. Notably, this work tackles MER only utilizing geometric
features, processed based on a graph model, which provides a new idea with much
higher efficiency to promote MER. The experimental results demonstrate that the
proposed method can achieve competitive or even superior performance with a
significantly reduced computational cost, and facial landmarks can
significantly contribute to MER and are worth further study for efficient ME
analysis.
Related papers
- Message-Passing Monte Carlo: Generating low-discrepancy point sets via Graph Neural Networks [64.39488944424095]
We present the first machine learning approach to generate low-discrepancy point sets named Message-Passing Monte Carlo points.
MPMC points are empirically shown to be either optimal or near-optimal with respect to the discrepancy for low dimension and small number of points.
arXiv Detail & Related papers (2024-05-23T21:17:20Z) - Semantic Graph Representation Learning for Handwritten Mathematical
Expression Recognition [57.60390958736775]
We propose a simple but efficient method to enhance semantic interaction learning (SIL)
We first construct a semantic graph based on the statistical symbol co-occurrence probabilities.
Then we design a semantic aware module (SAM), which projects the visual and classification feature into semantic space.
Our method achieves better recognition performance than prior arts on both CROHME and HME100K datasets.
arXiv Detail & Related papers (2023-08-21T06:23:41Z) - AU-aware graph convolutional network for Macro- and Micro-expression
spotting [44.507747407072685]
We propose a graph convolutional-based network, called Action-Unit-aWare Graph Convolutional Network (AUW-GCN)
To inject prior information and to cope with the problem of small datasets, AU-related statistics are encoded into the network.
Our results outperform baseline methods consistently and achieve new SOTA performance in two benchmark datasets.
arXiv Detail & Related papers (2023-03-16T07:00:36Z) - Prior Information based Decomposition and Reconstruction Learning for
Micro-Expression Recognition [45.46357824529522]
Prior information can guide the model to learn discriminative ME features effectively.
This paper proposes a Decomposition and Reconstruction-based Graph Representation Learning model to learn ME movement features.
arXiv Detail & Related papers (2023-03-03T08:34:28Z) - Discovering Generalizable Spatial Goal Representations via Graph-based
Active Reward Learning [17.58129740811116]
We propose a reward learning approach, Graph-based Equivalence Mappings (GEM)
GEM represents a spatial goal specification by a reward function conditioned on i) a graph indicating important spatial relationships between objects and ii) state equivalence mappings for each edge in the graph.
We show that GEM can drastically improve the generalizability of the learned goal representations over strong baselines.
arXiv Detail & Related papers (2022-11-24T18:59:06Z) - Motif-based Graph Representation Learning with Application to Chemical
Molecules [11.257235936629689]
Existing graph neural networks offer limited ability to capture complex interactions within local structural contexts.
We propose a new motif-based graph representation learning technique to better utilize local structural information.
MCM builds a motif vocabulary in an unsupervised way and deploys a novel motif convolution operation to extract the local structural context.
arXiv Detail & Related papers (2022-08-09T03:37:37Z) - Transferring Dual Stochastic Graph Convolutional Network for Facial
Micro-expression Recognition [7.62031665958404]
This paper presents a transferring dual Graph Convolutional Network (GCN) model.
We propose a graph construction method and dual graph convolutional network to extract more discriminative features from the micro-expression images.
Our proposed method achieves state-of-the-art performance on recently released MMEW benchmarks.
arXiv Detail & Related papers (2022-03-10T07:41:18Z) - Neural Topological SLAM for Visual Navigation [112.73876869904]
We design topological representations for space that leverage semantics and afford approximate geometric reasoning.
We describe supervised learning-based algorithms that can build, maintain and use such representations under noisy actuation.
arXiv Detail & Related papers (2020-05-25T17:56:29Z) - A Transfer Learning approach to Heatmap Regression for Action Unit
intensity estimation [50.261472059743845]
Action Units (AUs) are geometrically-based atomic facial muscle movements.
We propose a novel AU modelling problem that consists of jointly estimating their localisation and intensity.
A Heatmap models whether an AU occurs or not at a given spatial location.
arXiv Detail & Related papers (2020-04-14T16:51:13Z) - Spatial Pyramid Based Graph Reasoning for Semantic Segmentation [67.47159595239798]
We apply graph convolution into the semantic segmentation task and propose an improved Laplacian.
The graph reasoning is directly performed in the original feature space organized as a spatial pyramid.
We achieve comparable performance with advantages in computational and memory overhead.
arXiv Detail & Related papers (2020-03-23T12:28:07Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.