Prior Information based Decomposition and Reconstruction Learning for Micro-Expression Recognition

• URL: http://arxiv.org/abs/2303.01776v1
• Date: Fri, 3 Mar 2023 08:34:28 GMT
• Title: Prior Information based Decomposition and Reconstruction Learning for Micro-Expression Recognition
• Authors: Jinsheng Wei, Haoyu Chen, Guanming Lu, Jingjie Yan, Yue Xie and Guoying Zhao
• Abstract summary: Prior information can guide the model to learn discriminative ME features effectively. This paper proposes a Decomposition and Reconstruction-based Graph Representation Learning model to learn ME movement features.
• Score: 45.46357824529522
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Micro-expression recognition (MER) draws intensive research interest as micro-expressions (MEs) can infer genuine emotions. Prior information can guide the model to learn discriminative ME features effectively. However, most works focus on researching the general models with a stronger representation ability to adaptively aggregate ME movement information in a holistic way, which may ignore the prior information and properties of MEs. To solve this issue, driven by the prior information that the category of ME can be inferred by the relationship between the actions of facial different components, this work designs a novel model that can conform to this prior information and learn ME movement features in an interpretable way. Specifically, this paper proposes a Decomposition and Reconstruction-based Graph Representation Learning (DeRe-GRL) model to effectively learn high-level ME features. DeRe-GRL includes two modules: Action Decomposition Module (ADM) and Relation Reconstruction Module (RRM), where ADM learns action features of facial key components and RRM explores the relationship between these action features. Based on facial key components, ADM divides the geometric movement features extracted by the graph model-based backbone into several sub-features, and learns the map matrix to map these sub-features into multiple action features; then, RRM learns weights to weight all action features to build the relationship between action features. The experimental results demonstrate the effectiveness of the proposed modules, and the proposed method achieves competitive performance.

Related papers

• Micro-Expression Recognition by Motion Feature Extraction based on Pre-training [6.015288149235598]
  We propose a novel motion extraction strategy (MoExt) for the micro-expression recognition task. In MoExt, shape features and texture features are first extracted separately from onset and apex frames, and then motion features related to MEs are extracted based on shape features of both frames. The effectiveness of proposed method is validated on three commonly used datasets.
  arXiv  Detail & Related papers  (2024-07-10T03:51:34Z)
• An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition [49.45660055499103]
  Zero-shot human skeleton-based action recognition aims to construct a model that can recognize actions outside the categories seen during training. Previous research has focused on aligning sequences' visual and semantic spatial distributions. We introduce a new loss function sampling method to obtain a tight and robust representation.
  arXiv  Detail & Related papers  (2024-06-02T06:53:01Z)
• Exploring Graph-based Knowledge: Multi-Level Feature Distillation via Channels Relational Graph [8.646512035461994]
  In visual tasks, large teacher models capture essential features and deep information, enhancing performance. We propose a distillation framework based on graph knowledge, including a multi-level feature alignment strategy. We emphasize spectral embedding (SE) as a key technique in our distillation process, which merges the student's feature space with the relational knowledge and structural complexities similar to the teacher network.
  arXiv  Detail & Related papers  (2024-05-14T12:37:05Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• Named Entity and Relation Extraction with Multi-Modal Retrieval [51.660650522630526]
  Multi-modal named entity recognition (NER) and relation extraction (RE) aim to leverage relevant image information to improve the performance of NER and RE. We propose a novel Multi-modal Retrieval based framework (MoRe) MoRe contains a text retrieval module and an image-based retrieval module, which retrieve related knowledge of the input text and image in the knowledge corpus respectively.
  arXiv  Detail & Related papers  (2022-12-03T13:11:32Z)
• Tackling Visual Control via Multi-View Exploration Maximization [64.8463574294237]
  MEM is the first approach that combines multi-view representation learning and reward-driven exploration in reinforcement learning (RL) We evaluate MEM on various tasks from DeepMind Control Suite and Procgen games.
  arXiv  Detail & Related papers  (2022-11-28T11:29:56Z)
• Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition [38.579316014796945]
  Micro-expression recognition (MER) is valuable because the involuntary nature of micro-expressions (MEs) can reveal genuine emotions. This paper explores the contribution of facial landmarks and constructs a new framework to efficiently recognize MEs with sole facial landmark information. The experimental results demonstrate that the proposed method can achieve competitive or even superior performance with a significantly reduced computational cost.
  arXiv  Detail & Related papers  (2022-05-01T02:20:43Z)
• Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition [80.17419621762866]
  We propose a novel Feature Decomposition and Reconstruction Learning (FDRL) method for effective facial expression recognition. FDRL consists of two crucial networks: a Feature Decomposition Network (FDN) and a Feature Reconstruction Network (FRN)
  arXiv  Detail & Related papers  (2021-04-12T02:22:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.