Toward Motion Robustness: A masked attention regularization framework in remote photoplethysmography

• URL: http://arxiv.org/abs/2407.06653v1
• Date: Tue, 9 Jul 2024 08:25:30 GMT
• Title: Toward Motion Robustness: A masked attention regularization framework in remote photoplethysmography
• Authors: Pengfei Zhao, Qigong Sun, Xiaolin Tian, Yige Yang, Shuo Tao, Jie Cheng, Jiantong Chen,
• Abstract summary: MAR-r is a framework that integrates the impact of ROI localization and complex motion artifacts. MAR-r employs a masked attention regularization mechanism into the r field to capture semantic consistency of facial clips. It also employs a masking technique to prevent the model from overfitting on inaccurate ROIs and subsequently degrading its performance.
• Score: 5.743550396843244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: There has been growing interest in facial video-based remote photoplethysmography (rPPG) measurement recently, with a focus on assessing various vital signs such as heart rate and heart rate variability. Despite previous efforts on static datasets, their approaches have been hindered by inaccurate region of interest (ROI) localization and motion issues, and have shown limited generalization in real-world scenarios. To address these challenges, we propose a novel masked attention regularization (MAR-rPPG) framework that mitigates the impact of ROI localization and complex motion artifacts. Specifically, our approach first integrates a masked attention regularization mechanism into the rPPG field to capture the visual semantic consistency of facial clips, while it also employs a masking technique to prevent the model from overfitting on inaccurate ROIs and subsequently degrading its performance. Furthermore, we propose an enhanced rPPG expert aggregation (EREA) network as the backbone to obtain rPPG signals and attention maps simultaneously. Our EREA network is capable of discriminating divergent attentions from different facial areas and retaining the consistency of spatiotemporal attention maps. For motion robustness, a simple open source detector MediaPipe for data preprocessing is sufficient for our framework due to its superior capability of rPPG signal extraction and attention regularization. Exhaustive experiments on three benchmark datasets (UBFC-rPPG, PURE, and MMPD) substantiate the superiority of our proposed method, outperforming recent state-of-the-art works by a considerable margin.

Related papers

• FactorizePhys: Matrix Factorization for Multidimensional Attention in Remote Physiological Sensing [10.81951503398909]
  Factorized Self-Attention Module (FSAM) computes multidimensional attention from voxel embeddings using nonnegative matrix factorization. Our approach adeptly factorizes voxel embeddings to achieve comprehensive spatial, temporal, and channel attention, enhancing performance of generic signal extraction. FactorizePhys is an end-to-end 3D-CNN architecture for estimating blood volume pulse signals from raw video frames.
  arXiv  Detail & Related papers  (2024-11-03T12:22:58Z)
• SQUWA: Signal Quality Aware DNN Architecture for Enhanced Accuracy in Atrial Fibrillation Detection from Noisy PPG Signals [37.788535094404644]
  Atrial fibrillation (AF) significantly increases the risk of stroke, heart disease, and mortality. Photoplethysmography ( PPG) signals are susceptible to corruption from motion artifacts and other factors often encountered in ambulatory settings. We propose a novel deep learning model, designed to learn how to retain accurate predictions from partially corrupted PPG.
  arXiv  Detail & Related papers  (2024-04-15T01:07:08Z)
• Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection [96.539862328788]
  Existing forgery detection methods suffer from unsatisfactory generalization ability to determine the authenticity in the unseen domain. We propose a novel Attention Consistency Refined masked frequency forgery representation model toward generalizing face forgery detection algorithm (ACMF) Experiment results on several public face forgery datasets demonstrate the superior performance of the proposed method compared with the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-07-21T08:58:49Z)
• Sequential Attention Source Identification Based on Feature Representation [88.05527934953311]
  This paper proposes a sequence-to-sequence based localization framework called Temporal-sequence based Graph Attention Source Identification (TGASI) based on an inductive learning idea. It's worth mentioning that the inductive learning idea ensures that TGASI can detect the sources in new scenarios without knowing other prior knowledge.
  arXiv  Detail & Related papers  (2023-06-28T03:00:28Z)
• Mask Attack Detection Using Vascular-weighted Motion-robust rPPG Signals [21.884783786547782]
  R-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence. A landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. A lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features for classification.
  arXiv  Detail & Related papers  (2023-05-25T11:22:17Z)
• PS-ARM: An End-to-End Attention-aware Relation Mixer Network for Person Search [56.02761592710612]
  We propose a novel attention-aware relation mixer (ARM) for module person search. Our ARM module is native and does not rely on fine-grained supervision or topological assumptions. Our PS-ARM achieves state-of-the-art performance on both datasets.
  arXiv  Detail & Related papers  (2022-10-07T10:04:12Z)
• ScoreNet: Learning Non-Uniform Attention and Augmentation for Transformer-Based Histopathological Image Classification [11.680355561258427]
  High-resolution images hinder progress in digital pathology. patch-based processing often incorporates multiple instance learning (MIL) to aggregate local patch-level representations yielding image-level prediction. This paper proposes a transformer-based architecture specifically tailored for histological image classification. It combines fine-grained local attention with a coarse global attention mechanism to learn meaningful representations of high-resolution images at an efficient computational cost.
  arXiv  Detail & Related papers  (2022-02-15T16:55:09Z)
• Robust and Precise Facial Landmark Detection by Self-Calibrated Pose Attention Network [73.56802915291917]
  We propose a semi-supervised framework to achieve more robust and precise facial landmark detection. A Boundary-Aware Landmark Intensity (BALI) field is proposed to model more effective facial shape constraints. A Self-Calibrated Pose Attention (SCPA) model is designed to provide a self-learned objective function that enforces intermediate supervision.
  arXiv  Detail & Related papers  (2021-12-23T02:51:08Z)
• MC-LCR: Multi-modal contrastive classification by locally correlated representations for effective face forgery detection [11.124150983521158]
  We propose a novel framework named Multi-modal Contrastive Classification by Locally Correlated Representations. Our MC-LCR aims to amplify implicit local discrepancies between authentic and forged faces from both spatial and frequency domains. We achieve state-of-the-art performance and demonstrate the robustness and generalization of our method.
  arXiv  Detail & Related papers  (2021-10-07T09:24:12Z)
• Local Relation Learning for Face Forgery Detection [73.73130683091154]
  We propose a novel perspective of face forgery detection via local relation learning. Specifically, we propose a Multi-scale Patch Similarity Module (MPSM), which measures the similarity between features of local regions. We also propose an RGB-Frequency Attention Module (RFAM) to fuse information in both RGB and frequency domains for more comprehensive local feature representation.
  arXiv  Detail & Related papers  (2021-05-06T10:44:32Z)
• Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting [86.33696045574692]
  We propose a Cycle Consistency Panoptic Domain Adaptive Mask R-CNN (CyC-PDAM) architecture for unsupervised nuclei segmentation in histopathology images. We first propose a nuclei inpainting mechanism to remove the auxiliary generated objects in the synthesized images. Secondly, a semantic branch with a domain discriminator is designed to achieve panoptic-level domain adaptation.
  arXiv  Detail & Related papers  (2020-05-05T11:08:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.