Remote Heart Rate Monitoring in Smart Environments from Videos with Self-supervised Pre-training

• URL: http://arxiv.org/abs/2310.15388v1
• Date: Mon, 23 Oct 2023 22:41:04 GMT
• Title: Remote Heart Rate Monitoring in Smart Environments from Videos with Self-supervised Pre-training
• Authors: Divij Gupta, Ali Etemad
• Abstract summary: We introduce a solution that utilizes self-supervised contrastive learning for the estimation of remote photoplethys (mography) and heart rate monitoring. We propose the use of 3 spatial and 3 temporal augmentations for training an encoder through a contrastive framework, followed by utilizing the late-intermediate embeddings of the encoder for remote PPG and heart rate estimation.
• Score: 28.404118669462772
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in deep learning have made it increasingly feasible to estimate heart rate remotely in smart environments by analyzing videos. However, a notable limitation of deep learning methods is their heavy reliance on extensive sets of labeled data for effective training. To address this issue, self-supervised learning has emerged as a promising avenue. Building on this, we introduce a solution that utilizes self-supervised contrastive learning for the estimation of remote photoplethysmography (PPG) and heart rate monitoring, thereby reducing the dependence on labeled data and enhancing performance. We propose the use of 3 spatial and 3 temporal augmentations for training an encoder through a contrastive framework, followed by utilizing the late-intermediate embeddings of the encoder for remote PPG and heart rate estimation. Our experiments on two publicly available datasets showcase the improvement of our proposed approach over several related works as well as supervised learning baselines, as our results approach the state-of-the-art. We also perform thorough experiments to showcase the effects of using different design choices such as the video representation learning method, the augmentations used in the pre-training stage, and others. We also demonstrate the robustness of our proposed method over the supervised learning approaches on reduced amounts of labeled data.

Related papers

• Debiased Learning for Remote Sensing Data [29.794246747637104]
  We propose a highly effective semi-supervised approach tailored specifically to remote sensing data. First, we adapt the FixMatch framework to remote sensing data by designing robust strong and weak augmentations suitable for this domain. Second, we develop an effective semi-supervised learning method by removing bias in imbalanced training data resulting from both actual labels and pseudo-labels predicted by the model.
  arXiv  Detail & Related papers  (2023-12-24T03:33:30Z)
• Temporal Contrastive Learning with Curriculum [19.442685015494316]
  ConCur is a contrastive video representation learning method that uses curriculum learning to impose a dynamic sampling strategy. We conduct experiments on two popular action recognition datasets, UCF101 and HMDB51, on which our proposed method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-09-02T00:12:05Z)
• CCLF: A Contrastive-Curiosity-Driven Learning Framework for Sample-Efficient Reinforcement Learning [56.20123080771364]
  We develop a model-agnostic Contrastive-Curiosity-Driven Learning Framework (CCLF) for reinforcement learning. CCLF fully exploit sample importance and improve learning efficiency in a self-supervised manner. We evaluate this approach on the DeepMind Control Suite, Atari, and MiniGrid benchmarks.
  arXiv  Detail & Related papers  (2022-05-02T14:42:05Z)
• SURF: Semi-supervised Reward Learning with Data Augmentation for Feedback-efficient Preference-based Reinforcement Learning [168.89470249446023]
  We present SURF, a semi-supervised reward learning framework that utilizes a large amount of unlabeled samples with data augmentation. In order to leverage unlabeled samples for reward learning, we infer pseudo-labels of the unlabeled samples based on the confidence of the preference predictor. Our experiments demonstrate that our approach significantly improves the feedback-efficiency of the preference-based method on a variety of locomotion and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2022-03-18T16:50:38Z)
• The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video [10.479541955106328]
  The ability to reliably estimate physiological signals from video is a powerful tool in low-cost, pre-clinical health monitoring. We propose a new approach to remote photoplethysmography (r) - the measurement of blood volume changes from observations of a person's face or skin.
  arXiv  Detail & Related papers  (2021-11-18T15:21:33Z)
• Self-Supervised Learning Framework for Remote Heart Rate Estimation Using Spatiotemporal Augmentation [12.783744603679942]
  Recent deep learning methods have shown that heart rate can be measured remotely using facial videos. We present a 3D self-supervised learning framework for remote HR estimation on facial videos.
  arXiv  Detail & Related papers  (2021-07-16T04:00:13Z)
• Unsupervised Scale-consistent Depth Learning from Video [131.3074342883371]
  We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training. Thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system. The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training.
  arXiv  Detail & Related papers  (2021-05-25T02:17:56Z)
• Self-supervised Co-training for Video Representation Learning [103.69904379356413]
  We investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation training. We propose a novel self-supervised co-training scheme to improve the popular infoNCE loss. We evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval.
  arXiv  Detail & Related papers  (2020-10-19T17:59:01Z)
• Self-supervised Video Object Segmentation [76.83567326586162]
  The objective of this paper is self-supervised representation learning, with the goal of solving semi-supervised video object segmentation (a.k.a. dense tracking) We make the following contributions: (i) we propose to improve the existing self-supervised approach, with a simple, yet more effective memory mechanism for long-term correspondence matching; (ii) by augmenting the self-supervised approach with an online adaptation module, our method successfully alleviates tracker drifts caused by spatial-temporal discontinuity; (iv) we demonstrate state-of-the-art results among the self-supervised approaches on DAVIS-2017 and YouTube
  arXiv  Detail & Related papers  (2020-06-22T17:55:59Z)
• Auto-Rectify Network for Unsupervised Indoor Depth Estimation [119.82412041164372]
  We establish that the complex ego-motions exhibited in handheld settings are a critical obstacle for learning depth. We propose a data pre-processing method that rectifies training images by removing their relative rotations for effective learning. Our results outperform the previous unsupervised SOTA method by a large margin on the challenging NYUv2 dataset.
  arXiv  Detail & Related papers  (2020-06-04T08:59:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.