Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast

• URL: http://arxiv.org/abs/2208.04378v1
• Date: Mon, 8 Aug 2022 19:30:57 GMT
• Title: Contrast-Phys: Unsupervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast
• Authors: Zhaodong Sun, Xiaobai Li
• Abstract summary: Video-based remote physiological measurement face videos to measure the blood volume change signal, which is also called remote photoplethysmography (r) We use a 3DCNN model to generate multiple rtemporal signals from each video in different locations and train the model with a contrastive loss where r signals from the same video are pulled together while those from different videos are pushed away.
• Score: 17.691683039742323
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Video-based remote physiological measurement utilizes face videos to measure the blood volume change signal, which is also called remote photoplethysmography (rPPG). Supervised methods for rPPG measurements achieve state-of-the-art performance. However, supervised rPPG methods require face videos and ground truth physiological signals for model training. In this paper, we propose an unsupervised rPPG measurement method that does not require ground truth signals for training. We use a 3DCNN model to generate multiple rPPG signals from each video in different spatiotemporal locations and train the model with a contrastive loss where rPPG signals from the same video are pulled together while those from different videos are pushed away. We test on five public datasets, including RGB videos and NIR videos. The results show that our method outperforms the previous unsupervised baseline and achieves accuracies very close to the current best supervised rPPG methods on all five datasets. Furthermore, we also demonstrate that our approach can run at a much faster speed and is more robust to noises than the previous unsupervised baseline. Our code is available at https://github.com/zhaodongsun/contrast-phys.

Related papers

• SiNC+: Adaptive Camera-Based Vitals with Unsupervised Learning of Periodic Signals [6.458510829614774]
  We present the first non-contrastive unsupervised learning framework for signal regression. We find that encouraging sparse power spectra within normal physiological bandlimits and variance over batches of power spectra is sufficient for learning periodic signals.
  arXiv  Detail & Related papers  (2024-04-20T19:17:40Z)
• Refining Pre-Trained Motion Models [56.18044168821188]
  We take on the challenge of improving state-of-the-art supervised models with self-supervised training. We focus on obtaining a "clean" training signal from real-world unlabelled video. We show that our method yields reliable gains over fully-supervised methods in real videos.
  arXiv  Detail & Related papers  (2024-01-01T18:59:33Z)
• Contrast-Phys+: Unsupervised and Weakly-supervised Video-based Remote Physiological Measurement via Spatiotemporal Contrast [22.742875409103164]
  We propose Contrast-Phys+, a method that can be trained in both unsupervised and unsupervised settings. We employ a 3DCNN model to generate multiple rtemporal signals and incorporate prior knowledge of r into a contrastive loss function. Contrast-Phys+ outperforms the state-of-the-art supervised methods, even when using partially available or misaligned GT signals.
  arXiv  Detail & Related papers  (2023-09-13T12:50:21Z)
• Non-Contrastive Unsupervised Learning of Physiological Signals from Video [4.8327232174895745]
  We present the first non-contrastive unsupervised learning framework for signal regression to break free from labelled video data. With minimal assumptions of periodicity and finite bandwidth, our approach is capable of discovering blood volume pulse directly from unlabelled videos.
  arXiv  Detail & Related papers  (2023-03-14T14:34:51Z)
• Facial Video-based Remote Physiological Measurement via Self-supervised Learning [9.99375728024877]
  We introduce a novel framework that learns to estimate r signals from facial videos without the need of ground truth signals. Negative samples are generated via a learnable frequency module, which performs nonlinear signal frequency transformation. Next, we introduce a local r expert aggregation module to estimate r signals from augmented samples. It encodes complementary pulsation information from different face regions and aggregate them into one r prediction.
  arXiv  Detail & Related papers  (2022-10-27T13:03:23Z)
• Direct Dense Pose Estimation [138.56533828316833]
  Dense human pose estimation is the problem of learning dense correspondences between RGB images and the surfaces of human bodies. Prior dense pose estimation methods are all based on Mask R-CNN framework and operate in a top-down manner of first attempting to identify a bounding box for each person. We propose a novel alternative method for solving the dense pose estimation problem, called Direct Dense Pose (DDP)
  arXiv  Detail & Related papers  (2022-04-04T06:14:38Z)
• Deep Video Prior for Video Consistency and Propagation [58.250209011891904]
  We present a novel and general approach for blind video temporal consistency. Our method is only trained on a pair of original and processed videos directly instead of a large dataset. We show that temporal consistency can be achieved by training a convolutional neural network on a video with Deep Video Prior.
  arXiv  Detail & Related papers  (2022-01-27T16:38:52Z)
• PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer [55.936527926778695]
  Recent deep learning approaches focus on mining subtle r clues using convolutional neural networks with limited-temporal receptive fields. In this paper, we propose the PhysFormer, an end-to-end video transformer based architecture.
  arXiv  Detail & Related papers  (2021-11-23T18:57:11Z)
• The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video [10.479541955106328]
  The ability to reliably estimate physiological signals from video is a powerful tool in low-cost, pre-clinical health monitoring. We propose a new approach to remote photoplethysmography (r) - the measurement of blood volume changes from observations of a person's face or skin.
  arXiv  Detail & Related papers  (2021-11-18T15:21:33Z)
• Perceptual Learned Video Compression with Recurrent Conditional GAN [158.0726042755]
  We propose a Perceptual Learned Video Compression (PLVC) approach with recurrent conditional generative adversarial network. PLVC learns to compress video towards good perceptual quality at low bit-rate. The user study further validates the outstanding perceptual performance of PLVC in comparison with the latest learned video compression approaches.
  arXiv  Detail & Related papers  (2021-09-07T13:36:57Z)
• Self-Supervised Human Depth Estimation from Monocular Videos [99.39414134919117]
  Previous methods on estimating detailed human depth often require supervised training with ground truth' depth data. This paper presents a self-supervised method that can be trained on YouTube videos without known depth. Experiments demonstrate that our method enjoys better generalization and performs much better on data in the wild.
  arXiv  Detail & Related papers  (2020-05-07T09:45:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.