Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalization

• URL: http://arxiv.org/abs/2506.16160v1
• Date: Thu, 19 Jun 2025 09:17:30 GMT
• Title: Align the GAP: Prior-based Unified Multi-Task Remote Physiological Measurement Framework For Domain Generalization and Personalization
• Authors: Jiyao Wang, Xiao Yang, Hao Lu, Dengbo He, Kaishun Wu,
• Abstract summary: We proposed a unified framework for MSSDtextbfG and TTPtextbfPriors (textbfGAP) in biometrics and remote photoplesmography.<n>We expanded the MSSDG benchmark to the TTPA protocol on six publicly available datasets and introduced a new real-world driving dataset with complete labeling.
• Score: 13.53570294343287
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-source synsemantic domain generalization (MSSDG) for multi-task remote physiological measurement seeks to enhance the generalizability of these metrics and attracts increasing attention. However, challenges like partial labeling and environmental noise may disrupt task-specific accuracy. Meanwhile, given that real-time adaptation is necessary for personalized products, the test-time personalized adaptation (TTPA) after MSSDG is also worth exploring, while the gap between previous generalization and personalization methods is significant and hard to fuse. Thus, we proposed a unified framework for MSSD\textbf{G} and TTP\textbf{A} employing \textbf{P}riors (\textbf{GAP}) in biometrics and remote photoplethysmography (rPPG). We first disentangled information from face videos into invariant semantics, individual bias, and noise. Then, multiple modules incorporating priors and our observations were applied in different stages and for different facial information. Then, based on the different principles of achieving generalization and personalization, our framework could simultaneously address MSSDG and TTPA under multi-task remote physiological estimation with minimal adjustments. We expanded the MSSDG benchmark to the TTPA protocol on six publicly available datasets and introduced a new real-world driving dataset with complete labeling. Extensive experiments that validated our approach, and the codes along with the new dataset will be released.

Related papers

• Not Only Consistency: Enhance Test-Time Adaptation with Spatio-temporal Inconsistency for Remote Physiological Measurement [3.979038581055512]
  Remote photo signalsplesthysmography has emerged as a promising non-invasive method for monitoring the camera.<n>We propose a fully Test-Time Adaptation (TTA) strategy tailored for r tasks in this work.<n>Our method consistently outperforms existing techniques, presenting state-of-the-art performance in real-time self-text-supervised adaptation.
  arXiv  Detail & Related papers  (2025-07-10T16:39:49Z)
• PhysLLM: Harnessing Large Language Models for Cross-Modal Remote Physiological Sensing [49.243031514520794]
  Large Language Models (LLMs) excel at capturing long-range signals due to their text-centric design.<n>PhysLLM achieves state-the-art accuracy and robustness, demonstrating superior generalization across lighting variations and motion scenarios.
  arXiv  Detail & Related papers  (2025-05-06T15:18:38Z)
• Towards Generalizable Scene Change Detection [4.527270266697462]
  Current state-of-the-art Scene Change Detection approaches are unreliable under unseen environments and different temporal conditions.<n>We propose the Generalizable Scene Change Detection Framework (GeSCF) to address unseen domain performance and temporal consistency.<n>GeSCF achieves an average performance gain of 19.2% on existing SCD datasets and 30.0% on the ChangeVPR dataset, nearly doubling the prior art performance.
  arXiv  Detail & Related papers  (2024-09-10T04:45:25Z)
• Fully Test-Time rPPG Estimation via Synthetic Signal-Guided Feature Learning [8.901227918730562]
  TestTime Adaptation (TTA) enables the model to adaptively estimate r signals in various unseen domains by online adapting to unlabeled target data without referring to any source data. We develop a synthetic signal-guided feature learning method by pseudo r signals as pseudo ground truths to guide a conditional generator in generating latent r features.
  arXiv  Detail & Related papers  (2024-07-18T09:22:40Z)
• PeFAD: A Parameter-Efficient Federated Framework for Time Series Anomaly Detection [51.20479454379662]
  We propose a. Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.
  arXiv  Detail & Related papers  (2024-06-04T13:51:08Z)
• GenBench: A Benchmarking Suite for Systematic Evaluation of Genomic Foundation Models [56.63218531256961]
  We introduce GenBench, a benchmarking suite specifically tailored for evaluating the efficacy of Genomic Foundation Models. GenBench offers a modular and expandable framework that encapsulates a variety of state-of-the-art methodologies. We provide a nuanced analysis of the interplay between model architecture and dataset characteristics on task-specific performance.
  arXiv  Detail & Related papers  (2024-06-01T08:01:05Z)
• PhysMLE: Generalizable and Priors-Inclusive Multi-task Remote Physiological Measurement [24.424510759648072]
  This paper presents an end-to-end Mixture of Low-rank Experts for multi-task remote Physiological measurement (PhysMLE) PhysMLE is based on multiple low-rank experts with a novel router mechanism, enabling the model to adeptly handle both specifications and correlations within tasks. For fair and comprehensive evaluations, this paper proposed a large-scale multi-task generalization benchmark, named Multi-Source Synsemantic Domain Generalization protocol.
  arXiv  Detail & Related papers  (2024-05-10T02:36:54Z)
• Test-Time Domain Generalization for Face Anti-Spoofing [60.94384914275116]
  Face Anti-Spoofing (FAS) is pivotal in safeguarding facial recognition systems against presentation attacks. We introduce a novel Test-Time Domain Generalization framework for FAS, which leverages the testing data to boost the model's generalizability. Our method, consisting of Test-Time Style Projection (TTSP) and Diverse Style Shifts Simulation (DSSS), effectively projects the unseen data to the seen domain space.
  arXiv  Detail & Related papers  (2024-03-28T11:50:23Z)
• Neuron Structure Modeling for Generalizable Remote Physiological Measurement [35.33213338840912]
  Remote photoplethysmography (r) technology has drawn increasing attention in recent years. It can extract Blood Volume Pulse (BVP) from facial videos, making many applications more accessible. Existing methods struggle to generalize well for unseen domains. We propose a domain-label-free approach called NEuron STructure modeling (NEST)
  arXiv  Detail & Related papers  (2023-03-10T14:44:11Z)
• META: Mimicking Embedding via oThers' Aggregation for Generalizable Person Re-identification [68.39849081353704]
  Domain generalizable (DG) person re-identification (ReID) aims to test across unseen domains without access to the target domain data at training time. This paper presents a new approach called Mimicking Embedding via oThers' Aggregation (META) for DG ReID.
  arXiv  Detail & Related papers  (2021-12-16T08:06:50Z)
• PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer [55.936527926778695]
  Recent deep learning approaches focus on mining subtle r clues using convolutional neural networks with limited-temporal receptive fields. In this paper, we propose the PhysFormer, an end-to-end video transformer based architecture.
  arXiv  Detail & Related papers  (2021-11-23T18:57:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.