Video-based Remote Physiological Measurement via Cross-verified Feature
Disentangling
- URL: http://arxiv.org/abs/2007.08213v1
- Date: Thu, 16 Jul 2020 09:39:17 GMT
- Title: Video-based Remote Physiological Measurement via Cross-verified Feature
Disentangling
- Authors: Xuesong Niu, Zitong Yu, Hu Han, Xiaobai Li, Shiguang Shan, Guoying
Zhao
- Abstract summary: We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations.
We then use the distilled physiological features for robust multi-task physiological measurements.
The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
- Score: 121.50704279659253
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Remote physiological measurements, e.g., remote photoplethysmography (rPPG)
based heart rate (HR), heart rate variability (HRV) and respiration frequency
(RF) measuring, are playing more and more important roles under the application
scenarios where contact measurement is inconvenient or impossible. Since the
amplitude of the physiological signals is very small, they can be easily
affected by head movements, lighting conditions, and sensor diversities. To
address these challenges, we propose a cross-verified feature disentangling
strategy to disentangle the physiological features with non-physiological
representations, and then use the distilled physiological features for robust
multi-task physiological measurements. We first transform the input face videos
into a multi-scale spatial-temporal map (MSTmap), which can suppress the
irrelevant background and noise features while retaining most of the temporal
characteristics of the periodic physiological signals. Then we take pairwise
MSTmaps as inputs to an autoencoder architecture with two encoders (one for
physiological signals and the other for non-physiological information) and use
a cross-verified scheme to obtain physiological features disentangled with the
non-physiological features. The disentangled features are finally used for the
joint prediction of multiple physiological signals like average HR values and
rPPG signals. Comprehensive experiments on different large-scale public
datasets of multiple physiological measurement tasks as well as the
cross-database testing demonstrate the robustness of our approach.
Related papers
- PhysMLE: Generalizable and Priors-Inclusive Multi-task Remote Physiological Measurement [24.424510759648072]
This paper presents an end-to-end Mixture of Low-rank Experts for multi-task remote Physiological measurement (PhysMLE)
PhysMLE is based on multiple low-rank experts with a novel router mechanism, enabling the model to adeptly handle both specifications and correlations within tasks.
For fair and comprehensive evaluations, this paper proposed a large-scale multi-task generalization benchmark, named Multi-Source Synsemantic Domain Generalization protocol.
arXiv Detail & Related papers (2024-05-10T02:36:54Z) - Analyzing Participants' Engagement during Online Meetings Using Unsupervised Remote Photoplethysmography with Behavioral Features [50.82725748981231]
Engagement measurement finds application in healthcare, education, services.
Use of physiological and behavioral features is viable, but impracticality of traditional physiological measurement arises due to the need for contact sensors.
We demonstrate the feasibility of the unsupervised photoplethysmography (rmography) as an alternative for contact sensors.
arXiv Detail & Related papers (2024-04-05T20:39:16Z) - PhysFormer++: Facial Video-based Physiological Measurement with SlowFast
Temporal Difference Transformer [76.40106756572644]
Recent deep learning approaches focus on mining subtle clues using convolutional neural networks with limited-temporal receptive fields.
In this paper, we propose two end-to-end video transformer based on PhysFormer and Phys++++, to adaptively aggregate both local and global features for r representation enhancement.
Comprehensive experiments are performed on four benchmark datasets to show our superior performance on both intra-temporal and cross-dataset testing.
arXiv Detail & Related papers (2023-02-07T15:56:03Z) - PulseImpute: A Novel Benchmark Task for Pulsative Physiological Signal
Imputation [54.839600943189915]
Mobile Health (mHealth) is the ability to use wearable sensors to monitor participant physiology at high frequencies during daily life to enable temporally-precise health interventions.
Despite a rich imputation literature, existing techniques are ineffective for the pulsative signals which comprise many mHealth applications.
We address this gap with PulseImpute, the first large-scale pulsative signal imputation challenge which includes realistic mHealth missingness models, an extensive set of baselines, and clinically-relevant downstream tasks.
arXiv Detail & Related papers (2022-12-14T21:39:15Z) - DRNet: Decomposition and Reconstruction Network for Remote Physiological
Measurement [39.73408626273354]
Existing methods are generally divided into two groups.
The first focuses on mining the subtle volume pulse (BVP) signals from face videos, but seldom explicitly models the noises that dominate face video content.
The second focuses on modeling noisy data directly, resulting in suboptimal performance due to the lack of regularity of these severe random noises.
arXiv Detail & Related papers (2022-06-12T07:40:10Z) - SCAMPS: Synthetics for Camera Measurement of Physiological Signals [17.023803380199492]
We present SCAMPS, a dataset of synthetics containing 2,800 videos (1.68M frames) with aligned cardiac and respiratory signals and facial action intensities.
We provide descriptive statistics about the underlying waveforms, including inter-beat interval, heart rate variability, and pulse arrival time.
arXiv Detail & Related papers (2022-06-08T23:48:41Z) - PhysFormer: Facial Video-based Physiological Measurement with Temporal
Difference Transformer [55.936527926778695]
Recent deep learning approaches focus on mining subtle r clues using convolutional neural networks with limited-temporal receptive fields.
In this paper, we propose the PhysFormer, an end-to-end video transformer based architecture.
arXiv Detail & Related papers (2021-11-23T18:57:11Z) - Non-contact PPG Signal and Heart Rate Estimation with Multi-hierarchical
Convolutional Network [12.119293125608976]
Heart rate (HR) are important physiological parameters of the human body.
This study presents an efficient multi-archhierical- convolutional network that can estimate HR from face video clips.
arXiv Detail & Related papers (2021-04-06T03:04:27Z) - Discriminative Singular Spectrum Classifier with Applications on
Bioacoustic Signal Recognition [67.4171845020675]
We present a bioacoustic signal classifier equipped with a discriminative mechanism to extract useful features for analysis and classification efficiently.
Unlike current bioacoustic recognition methods, which are task-oriented, the proposed model relies on transforming the input signals into vector subspaces.
The validity of the proposed method is verified using three challenging bioacoustic datasets containing anuran, bee, and mosquito species.
arXiv Detail & Related papers (2021-03-18T11:01:21Z) - Learning Generalizable Physiological Representations from Large-scale
Wearable Data [12.863826659440026]
We present a novel self-supervised representation learning method using activity and heart rate (HR) signals without semantic labels.
We show that the resulting embeddings can generalize in various downstream tasks through transfer learning with linear classifiers.
Overall, we propose the first multimodal self-supervised method for behavioral and physiological data with implications for large-scale health and lifestyle monitoring.
arXiv Detail & Related papers (2020-11-09T17:56:03Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.