A Multi-stream Convolutional Neural Network for Micro-expression
Recognition Using Optical Flow and EVM
- URL: http://arxiv.org/abs/2011.03756v2
- Date: Tue, 10 Nov 2020 10:34:27 GMT
- Title: A Multi-stream Convolutional Neural Network for Micro-expression
Recognition Using Optical Flow and EVM
- Authors: Jinming Liu, Ke Li, Baolin Song, Li Zhao
- Abstract summary: Micro-expression (ME) recognition plays a crucial role in a wide range of applications, particularly in public security and psychotherapy.
Recently, traditional methods rely excessively on machine learning design and the recognition rate is not high enough for its practical application.
We design a multi-stream convolutional neural network (MSCNN) for ME recognition in this paper.
- Score: 7.511596258731931
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Micro-expression (ME) recognition plays a crucial role in a wide range of
applications, particularly in public security and psychotherapy. Recently,
traditional methods rely excessively on machine learning design and the
recognition rate is not high enough for its practical application because of
its short duration and low intensity. On the other hand, some methods based on
deep learning also cannot get high accuracy due to problems such as the
imbalance of databases. To address these problems, we design a multi-stream
convolutional neural network (MSCNN) for ME recognition in this paper.
Specifically, we employ EVM and optical flow to magnify and visualize subtle
movement changes in MEs and extract the masks from the optical flow images. And
then, we add the masks, optical flow images, and grayscale images into the
MSCNN. After that, in order to overcome the imbalance of databases, we added a
random over-sampler after the Dense Layer of the neural network. Finally,
extensive experiments are conducted on two public ME databases: CASME II and
SAMM. Compared with many recent state-of-the-art approaches, our method
achieves more promising recognition results.
Related papers
- Scaling Efficient Masked Autoencoder Learning on Large Remote Sensing Dataset [66.15872913664407]
This study introduces textbfRS-4M, a large-scale dataset designed to enable highly efficient MIM training on RS images.
We propose an efficient MIM method, termed textbfSelectiveMAE, which dynamically encodes and reconstructs a subset of patch tokens selected based on their semantic richness.
Experiments show that SelectiveMAE significantly boosts training efficiency by 2.2-2.7 times and enhances the classification, detection, and segmentation performance of the baseline MIM model.
arXiv Detail & Related papers (2024-06-17T15:41:57Z) - MindFormer: A Transformer Architecture for Multi-Subject Brain Decoding via fMRI [50.55024115943266]
We introduce a new Transformer architecture called MindFormer to generate fMRI-conditioned feature vectors.
MindFormer incorporates two key innovations: 1) a novel training strategy based on the IP-Adapter to extract semantically meaningful features from fMRI signals, and 2) a subject specific token and linear layer that effectively capture individual differences in fMRI signals.
arXiv Detail & Related papers (2024-05-28T00:36:25Z) - Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement
Learning [53.00683059396803]
Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images.
We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy.
Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
arXiv Detail & Related papers (2023-10-06T10:40:46Z) - Disruptive Autoencoders: Leveraging Low-level features for 3D Medical
Image Pre-training [51.16994853817024]
This work focuses on designing an effective pre-training framework for 3D radiology images.
We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations.
The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
arXiv Detail & Related papers (2023-07-31T17:59:42Z) - Deep Multi-Threshold Spiking-UNet for Image Processing [51.88730892920031]
This paper introduces the novel concept of Spiking-UNet for image processing, which combines the power of Spiking Neural Networks (SNNs) with the U-Net architecture.
To achieve an efficient Spiking-UNet, we face two primary challenges: ensuring high-fidelity information propagation through the network via spikes and formulating an effective training strategy.
Experimental results show that, on image segmentation and denoising, our Spiking-UNet achieves comparable performance to its non-spiking counterpart.
arXiv Detail & Related papers (2023-07-20T16:00:19Z) - LWGNet: Learned Wirtinger Gradients for Fourier Ptychographic Phase
Retrieval [14.588976801396576]
We propose a hybrid model-driven residual network that combines the knowledge of the forward imaging system with a deep data-driven network.
Unlike other conventional unrolling techniques, LWGNet uses fewer stages while performing at par or even better than existing traditional and deep learning techniques.
This improvement in performance for low-bit depth and low-cost sensors has the potential to bring down the cost of FPM imaging setup significantly.
arXiv Detail & Related papers (2022-08-08T17:22:54Z) - SiPRNet: End-to-End Learning for Single-Shot Phase Retrieval [8.820823270160695]
convolutional neural networks (CNN) have played important roles in various image reconstruction tasks.
In this paper, we design a novel CNN structure, named SiPRNet, to recover a signal from a single Fourier intensity measurement.
The proposed approach consistently outperforms other CNN-based and traditional optimization-based methods in single-shot maskless phase retrieval.
arXiv Detail & Related papers (2022-05-23T16:24:52Z) - Intelligent Masking: Deep Q-Learning for Context Encoding in Medical
Image Analysis [48.02011627390706]
We develop a novel self-supervised approach that occludes targeted regions to improve the pre-training procedure.
We show that training the agent against the prediction model can significantly improve the semantic features extracted for downstream classification tasks.
arXiv Detail & Related papers (2022-03-25T19:05:06Z) - Transferring Dual Stochastic Graph Convolutional Network for Facial
Micro-expression Recognition [7.62031665958404]
This paper presents a transferring dual Graph Convolutional Network (GCN) model.
We propose a graph construction method and dual graph convolutional network to extract more discriminative features from the micro-expression images.
Our proposed method achieves state-of-the-art performance on recently released MMEW benchmarks.
arXiv Detail & Related papers (2022-03-10T07:41:18Z) - Multiscale Convolutional Transformer with Center Mask Pretraining for
Hyperspectral Image Classificationtion [14.33259265286265]
We propose a noval multi-scale convolutional embedding module for hyperspectral images (HSI) to realize effective extraction of spatial-spectral information.
Similar to Mask autoencoder, but our pre-training method only masks the corresponding token of the central pixel in the encoder, and inputs the remaining token into the decoder to reconstruct the spectral information of the central pixel.
arXiv Detail & Related papers (2022-03-09T14:42:26Z) - Deep Residual Dense U-Net for Resolution Enhancement in Accelerated MRI
Acquisition [19.422926534305837]
We propose a deep-learning approach, aiming at reconstructing high-quality images from accelerated MRI acquisition.
Specifically, we use Convolutional Neural Network (CNN) to learn the differences between the aliased images and the original images.
Considering the peculiarity of the down-sampled k-space data, we introduce a new term to the loss function in learning, which effectively employs the given k-space data.
arXiv Detail & Related papers (2020-01-13T19:01:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.