Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces

• URL: http://arxiv.org/abs/2202.04101v3
• Date: Thu, 4 May 2023 18:25:35 GMT
• Title: Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces
• Authors: Constantino \'Alvarez Casado and Miguel Bordallo L\'opez
• Abstract summary: Photoplethys signals have become a key technology in many fields, such as medicine, well-being, or sports. Our work proposes a set of pipelines to extract PPG signals from the face robustly, reliably, and robustness.
• Score: 0.456877715768796
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Photoplethysmography (PPG) signals have become a key technology in many fields, such as medicine, well-being, or sports. Our work proposes a set of pipelines to extract remote PPG signals (rPPG) from the face robustly, reliably, and configurable. We identify and evaluate the possible choices in the critical steps of unsupervised rPPG methodologies. We assess a state-of-the-art processing pipeline in six different datasets, incorporating important corrections in the methodology that ensure reproducible and fair comparisons. In addition, we extend the pipeline by proposing three novel ideas; 1) a new method to stabilize the detected face based on a rigid mesh normalization; 2) a new method to dynamically select the different regions in the face that provide the best raw signals, and 3) a new RGB to rPPG transformation method, called Orthogonal Matrix Image Transformation (OMIT) based on QR decomposition, that increases robustness against compression artifacts. We show that all three changes introduce noticeable improvements in retrieving rPPG signals from faces, obtaining state-of-the-art results compared with unsupervised, non-learning-based methodologies and, in some databases, very close to supervised, learning-based methods. We perform a comparative study to quantify the contribution of each proposed idea. In addition, we depict a series of observations that could help in future implementations.

Related papers

• Deep Learning Based Speckle Filtering for Polarimetric SAR Images. Application to Sentinel-1 [51.404644401997736]
  We propose a complete framework to remove speckle in polarimetric SAR images using a convolutional neural network. Experiments show that the proposed approach offers exceptional results in both speckle reduction and resolution preservation.
  arXiv  Detail & Related papers  (2024-08-28T10:07:17Z)
• DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation [8.901227918730562]
  Photoplethysvolution (rmography) aims to measure physiological signals and Heart Rate (HR) from facial videos. Recent unsupervised r estimation methods have shown promising potential in estimating r signals from facial regions without relying on ground truth r signals. We propose a novel Deinterfered and Descriptive r Estimation Network (DD-rNet) to eliminate the interference within r features for learning genuine r signals.
  arXiv  Detail & Related papers  (2024-07-31T07:43:58Z)
• 3D Human Pose Analysis via Diffusion Synthesis [65.268245109828]
  PADS represents the first diffusion-based framework for tackling general 3D human pose analysis within the inverse problem framework. Its performance has been validated on different benchmarks, signaling the adaptability and robustness of this pipeline.
  arXiv  Detail & Related papers  (2024-01-17T02:59:34Z)
• Mask Attack Detection Using Vascular-weighted Motion-robust rPPG Signals [21.884783786547782]
  R-based face anti-spoofing methods often suffer from performance degradation due to unstable face alignment in the video sequence. A landmark-anchored face stitching method is proposed to align the faces robustly and precisely at the pixel-wise level by using both SIFT keypoints and facial landmarks. A lightweight EfficientNet with a Gated Recurrent Unit (GRU) is designed to extract both spatial and temporal features for classification.
  arXiv  Detail & Related papers  (2023-05-25T11:22:17Z)
• Non-Contact Heart Rate Measurement from Deteriorated Videos [0.3149883354098941]
  Remote photoplethysmography (rmography) offers a state-of-the-art, non-contact methodology for estimating human pulse by analyzing facial videos. In this study, we apply image processing to intentionally degrade video quality, mimicking challenging conditions. Our results reveal a significant decrease in accuracy in the presence of these artifacts, prompting us to propose the application of restoration techniques.
  arXiv  Detail & Related papers  (2023-04-28T11:58:36Z)
• Learning Motion-Robust Remote Photoplethysmography through Arbitrary Resolution Videos [31.512551653273373]
  In the real-world long-term health monitoring scenario, the distance of participants and their head movements usually vary by time, resulting in the inaccurate r measurement. Different from the previous r models designed for a constant distance between camera and participants, in this paper, we propose two plug-and-play blocks (i.e., physiological signal feature extraction block (PFE) and temporal face alignment block (TFA)) to alleviate the degradation of changing distance and head motion.
  arXiv  Detail & Related papers  (2022-11-30T11:50:08Z)
• Consistency Regularization for Deep Face Anti-Spoofing [69.70647782777051]
  Face anti-spoofing (FAS) plays a crucial role in securing face recognition systems. Motivated by this exciting observation, we conjecture that encouraging feature consistency of different views may be a promising way to boost FAS models. We enhance both Embedding-level and Prediction-level Consistency Regularization (EPCR) in FAS.
  arXiv  Detail & Related papers  (2021-11-24T08:03:48Z)
• The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video [10.479541955106328]
  The ability to reliably estimate physiological signals from video is a powerful tool in low-cost, pre-clinical health monitoring. We propose a new approach to remote photoplethysmography (r) - the measurement of blood volume changes from observations of a person's face or skin.
  arXiv  Detail & Related papers  (2021-11-18T15:21:33Z)
• 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop [128.07841893637337]
  Regression-based methods have recently shown promising results in reconstructing human meshes from monocular images. Minor deviation in parameters may lead to noticeable misalignment between the estimated meshes and image evidences. We propose a Pyramidal Mesh Alignment Feedback (PyMAF) loop to leverage a feature pyramid and rectify the predicted parameters.
  arXiv  Detail & Related papers  (2021-03-30T17:07:49Z)
• Deep Shells: Unsupervised Shape Correspondence with Optimal Transport [52.646396621449]
  We propose a novel unsupervised learning approach to 3D shape correspondence. We show that the proposed method significantly improves over the state-of-the-art on multiple datasets.
  arXiv  Detail & Related papers  (2020-10-28T22:24:07Z)
• Towards High Performance Human Keypoint Detection [87.1034745775229]
  We find that context information plays an important role in reasoning human body configuration and invisible keypoints. Inspired by this, we propose a cascaded context mixer ( CCM) which efficiently integrates spatial and channel context information. To maximize CCM's representation capability, we develop a hard-negative person detection mining strategy and a joint-training strategy. We present several sub-pixel refinement techniques for postprocessing keypoint predictions to improve detection accuracy.
  arXiv  Detail & Related papers  (2020-02-03T02:24:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.