Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos

• URL: http://arxiv.org/abs/2106.06489v1
• Date: Fri, 11 Jun 2021 16:19:48 GMT
• Title: Shallow Optical Flow Three-Stream CNN for Macro- and Micro-Expression Spotting from Long Videos
• Authors: Gen-Bing Liong, John See, Lai-Kuan Wong
• Abstract summary: We propose a model to predict a score that captures the likelihood of a frame being in an expression interval. We demonstrate the efficacy and efficiency of the proposed approach on the recent MEGC 2020 benchmark.
• Score: 15.322908569777551
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Facial expressions vary from the visible to the subtle. In recent years, the analysis of micro-expressions $-$ a natural occurrence resulting from the suppression of one's true emotions, has drawn the attention of researchers with a broad range of potential applications. However, spotting microexpressions in long videos becomes increasingly challenging when intertwined with normal or macro-expressions. In this paper, we propose a shallow optical flow three-stream CNN (SOFTNet) model to predict a score that captures the likelihood of a frame being in an expression interval. By fashioning the spotting task as a regression problem, we introduce pseudo-labeling to facilitate the learning process. We demonstrate the efficacy and efficiency of the proposed approach on the recent MEGC 2020 benchmark, where state-of-the-art performance is achieved on CAS(ME)$^{2}$ with equally promising results on SAMM Long Videos.

Related papers

• SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting [11.978551396144532]
  In this paper, we propose an efficient framework for facial expression spotting. First, we propose a Sliding Window-based Multi-Resolution Optical flow (SW-MRO) feature, which calculates multi-resolution optical flow of the input sequence within compact sliding windows. Second, we propose SpotFormer, a multi-scale-temporal Transformer that simultaneously encodes facial-temporal relationships of the SW-MRO features for accurate frame-level probability estimation. Third, we introduce supervised contrastive learning into SpotFormer to enhance the discriminability between different types of expressions.
  arXiv  Detail & Related papers  (2024-07-30T13:02:08Z)
• Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition [48.21696443824074]
  We propose a novel framework for micro-expression recognition, named the Adaptive Temporal Motion Guided Graph Convolution Network (ATM-GCN) Our framework excels at capturing temporal dependencies between frames across the entire clip, thereby enhancing micro-expression recognition at the clip level.
  arXiv  Detail & Related papers  (2024-06-13T10:57:24Z)
• From Macro to Micro: Boosting micro-expression recognition via pre-training on macro-expression videos [9.472210792839023]
  Micro-expression recognition (MER) has drawn increasing attention in recent years due to its potential applications in intelligent medical and lie detection. We propose a generalized transfer learning paradigm, called textbfMAcro-expression textbfTO textbfMIcro-expression (MA2MI) Under our paradigm, networks can learns the ability to represent subtle facial movement by reconstructing future frames.
  arXiv  Detail & Related papers  (2024-05-26T06:42:06Z)
• RMES: Real-Time Micro-Expression Spotting Using Phase From Riesz Pyramid [4.449835214520728]
  Micro-expressions (MEs) are involuntary and subtle facial expressions that are thought to reveal feelings people are trying to hide. Recent works leverage detailed facial motion representations, such as the optical flow, leading to high computational complexity. We propose RMES, a real-time ME spotting framework, to reduce computational complexity and achieve real-time operation.
  arXiv  Detail & Related papers  (2023-05-09T15:22:18Z)
• Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models [96.76758318732308]
  We show that the recently proposed Deep Equilibrium Model (DEQ) can be naturally adapted to this form of computation. Our Landmark DEQ (LDEQ) achieves state-of-the-art performance on the WFLW facial landmark dataset.
  arXiv  Detail & Related papers  (2023-04-02T19:08:02Z)
• Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms [52.58031087639394]
  micro-expressions are involuntary and transient facial expressions. They can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, their detection and recognition is difficult and relies heavily on expert experiences.
  arXiv  Detail & Related papers  (2022-01-30T05:14:13Z)
• Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition [61.374467942519374]
  We propose a novel a-temporal transformer architecture -- to the best of our knowledge, the first purely transformer based approach for micro-expression recognition. The architecture comprises a spatial encoder which learns spatial patterns, a temporal dimension classification for temporal analysis, and a head. A comprehensive evaluation on three widely used spontaneous micro-expression data sets, shows that the proposed approach consistently outperforms the state of the art.
  arXiv  Detail & Related papers  (2021-12-10T22:10:31Z)
• Efficient Lightweight 3D-CNN using Frame Skipping and Contrast Enhancement for Facial Macro- and Micro-expression Spotting [8.249165772349127]
  We construct an efficient lightweight 3D-Convolutional Network using Frame Skipping and Contrast Enhancement (EL-FACE) for the micro-expression spotting task. Our model achieves state-of-the-art performance in SAMM Long Videos and remained competitive in the CAS(ME)2 dataset.
  arXiv  Detail & Related papers  (2021-05-13T14:55:06Z)
• An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge [24.495792982803124]
  Facial micro-expressions indicate brief and subtle facial movements that appear during emotional communication. In recent years, micro-expression recognition (MER) has drawn much attention because it can benefit a wide range of applications. MER approaches from three novel aspects: macro-to-micro adaptation, recognition based on key apex frames, and recognition based on facial action units.
  arXiv  Detail & Related papers  (2020-12-21T13:20:17Z)
• Video-based Facial Expression Recognition using Graph Convolutional Networks [57.980827038988735]
  We introduce a Graph Convolutional Network (GCN) layer into a common CNN-RNN based model for video-based facial expression recognition. We evaluate our method on three widely-used datasets, CK+, Oulu-CASIA and MMI, and also one challenging wild dataset AFEW8.0.
  arXiv  Detail & Related papers  (2020-10-26T07:31:51Z)
• Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition [86.31412529187243]
  Few-shot video recognition aims at learning new actions with only very few labeled samples. We propose a depth guided Adaptive Meta-Fusion Network for few-shot video recognition which is termed as AMeFu-Net.
  arXiv  Detail & Related papers  (2020-10-20T03:06:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.