Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning
- URL: http://arxiv.org/abs/2407.13322v2
- Date: Mon, 22 Jul 2024 05:47:36 GMT
- Title: Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning
- Authors: Pei-Kai Huang, Tzu-Hsien Chen, Ya-Ting Chan, Kuan-Wen Chen, Chiou-Ting Hsu,
- Abstract summary: Many remote photoplethysmography (r) estimation models have achieved promising performance on the training domain but often fail to measure the physiological signals or heart rates (HR) on test domains.
In r estimation problems, the adapted model usually confronts challenges of estimating target data with various domain information.
TestTime-Time Adaptation (TTA), online adapting to unlabeled target data without referring to any source data, enables the model to adaptively estimate r signals of various unseen domains.
- Score: 8.901227918730562
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Many remote photoplethysmography (rPPG) estimation models have achieved promising performance on the training domain but often fail to measure the physiological signals or heart rates (HR) on test domains. Domain generalization (DG) or domain adaptation (DA) techniques are therefore adopted in the offline training stage to adapt the model to the unobserved or observed test domain by referring to all the available source domain data. However, in rPPG estimation problems, the adapted model usually confronts challenges of estimating target data with various domain information, such as different video capturing settings, individuals of different age ranges, or of different HR distributions. In contrast, Test-Time Adaptation (TTA), by online adapting to unlabeled target data without referring to any source data, enables the model to adaptively estimate rPPG signals of various unseen domains. In this paper, we first propose a novel TTA-rPPG benchmark, which encompasses various domain information and HR distributions, to simulate the challenges encountered in rPPG estimation. Next, we propose a novel synthetic signal-guided rPPG estimation framework with a two-fold purpose. First, we design an effective spectral-based entropy minimization to enforce the rPPG model to learn new target domain information. Second, we develop a synthetic signal-guided feature learning, by synthesizing pseudo rPPG signals as pseudo ground-truths to guide a conditional generator to generate latent rPPG features. The synthesized rPPG signals and the generated rPPG features are used to guide the rPPG model to broadly cover various HR distributions. Our extensive experiments on the TTA-rPPG benchmark show that the proposed method achieves superior performance and outperforms previous DG and DA methods across most protocols of the proposed TTA-rPPG benchmark.
Related papers
- Continual Learning for Remote Physiological Measurement: Minimize Forgetting and Simplify Inference [4.913049603343811]
Existing r measurement methods often overlook the incremental learning scenario.
Most existing class incremental learning approaches are unsuitable for r measurement.
We present a novel method named ADDP to tackle continual learning for r measurement.
arXiv Detail & Related papers (2024-07-19T01:49:09Z) - Improving Out-of-Distribution Generalization of Trajectory Prediction for Autonomous Driving via Polynomial Representations [16.856874154363588]
We present an OoD testing protocol that homogenizes datasets and prediction tasks across two large-scale motion datasets.
With a much smaller model size, training effort, and inference time, we reach near SotA performance for ID testing and significantly improve robustness in OoD testing.
arXiv Detail & Related papers (2024-07-18T12:00:32Z) - Test-Time Domain Generalization for Face Anti-Spoofing [60.94384914275116]
Face Anti-Spoofing (FAS) is pivotal in safeguarding facial recognition systems against presentation attacks.
We introduce a novel Test-Time Domain Generalization framework for FAS, which leverages the testing data to boost the model's generalizability.
Our method, consisting of Test-Time Style Projection (TTSP) and Diverse Style Shifts Simulation (DSSS), effectively projects the unseen data to the seen domain space.
arXiv Detail & Related papers (2024-03-28T11:50:23Z) - Deep adaptative spectral zoom for improved remote heart rate estimation [10.220888127527152]
Chirp-Z Transform (CZT) can refine the spectrum to the narrow-band range of interest for heart rate, providing improved frequential resolution and, consequently, more accurate estimation.
This paper presents the advantages of employing the CZT for remote HR estimation and introduces a novel data-driven adaptive CZT estimator.
arXiv Detail & Related papers (2024-03-11T16:55:19Z) - Consistency Regularization for Generalizable Source-free Domain
Adaptation [62.654883736925456]
Source-free domain adaptation (SFDA) aims to adapt a well-trained source model to an unlabelled target domain without accessing the source dataset.
Existing SFDA methods ONLY assess their adapted models on the target training set, neglecting the data from unseen but identically distributed testing sets.
We propose a consistency regularization framework to develop a more generalizable SFDA method.
arXiv Detail & Related papers (2023-08-03T07:45:53Z) - GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP,
and Beyond [101.5329678997916]
We study sample efficient reinforcement learning (RL) under the general framework of interactive decision making.
We propose a novel complexity measure, generalized eluder coefficient (GEC), which characterizes the fundamental tradeoff between exploration and exploitation.
We show that RL problems with low GEC form a remarkably rich class, which subsumes low Bellman eluder dimension problems, bilinear class, low witness rank problems, PO-bilinear class, and generalized regular PSR.
arXiv Detail & Related papers (2022-11-03T16:42:40Z) - BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy
for Source-free Domain Adaptation [74.93176783541332]
Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data.
To make up for the absence of source data, most existing methods introduced feature prototype based pseudo-labeling strategies.
We propose a general class-Balanced Multicentric Dynamic prototype strategy for the SFDA task.
arXiv Detail & Related papers (2022-04-06T13:23:02Z) - Face2PPG: An unsupervised pipeline for blood volume pulse extraction
from faces [0.456877715768796]
Photoplethys signals have become a key technology in many fields, such as medicine, well-being, or sports.
Our work proposes a set of pipelines to extract PPG signals from the face robustly, reliably, and robustness.
arXiv Detail & Related papers (2022-02-08T19:06:20Z) - Graphical Modeling for Multi-Source Domain Adaptation [56.05348879528149]
Multi-Source Domain Adaptation (MSDA) focuses on transferring the knowledge from multiple source domains to the target domain.
We propose two types of graphical models,i.e. Conditional Random Field for MSDA (CRF-MSDA) and Markov Random Field for MSDA (MRF-MSDA)
We evaluate these two models on four standard benchmark data sets of MSDA with distinct domain shift and data complexity.
arXiv Detail & Related papers (2021-04-27T09:04:22Z) - Semi-Supervised Domain Adaptation with Prototypical Alignment and
Consistency Learning [86.6929930921905]
This paper studies how much it can help address domain shifts if we further have a few target samples labeled.
To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks.
Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
arXiv Detail & Related papers (2021-04-19T08:46:08Z) - Variational Temporal Deep Generative Model for Radar HRRP Target
Recognition [39.01318281591659]
We develop a recurrent gamma belief network (rGBN) for radar automatic target recognition (RATR) based on high-resolution range profile (HRRP)
The proposed rGBN adopts a hierarchy of gamma distributions to build its temporal deep generative model.
Experimental results on synthetic and measured HRRP data show that the proposed models are efficient in computation, have good classification accuracy and ability, and provide highly interpretable multi-stochastic-layer latent structure.
arXiv Detail & Related papers (2020-09-28T02:03:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.