Training Robust Deep Physiological Measurement Models with Synthetic Video-based Data

• URL: http://arxiv.org/abs/2311.05371v2
• Date: Wed, 15 Nov 2023 13:57:53 GMT
• Title: Training Robust Deep Physiological Measurement Models with Synthetic Video-based Data
• Authors: Yuxuan Ou, Yuzhe Zhang, Yuntang Wang, Shwetak Patel, Daniel McDuf, Yuzhe Yang, Xin Liu
• Abstract summary: We propose measures to add real-world noise to synthetic physiological signals and corresponding facial videos. Our results show that we were able to reduce the average MAE from 6.9 to 2.0.
• Score: 11.31971398273479
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in supervised deep learning techniques have demonstrated the possibility to remotely measure human physiological vital signs (e.g., photoplethysmograph, heart rate) just from facial videos. However, the performance of these methods heavily relies on the availability and diversity of real labeled data. Yet, collecting large-scale real-world data with high-quality labels is typically challenging and resource intensive, which also raises privacy concerns when storing personal bio-metric data. Synthetic video-based datasets (e.g., SCAMPS \cite{mcduff2022scamps}) with photo-realistic synthesized avatars are introduced to alleviate the issues while providing high-quality synthetic data. However, there exists a significant gap between synthetic and real-world data, which hinders the generalization of neural models trained on these synthetic datasets. In this paper, we proposed several measures to add real-world noise to synthetic physiological signals and corresponding facial videos. We experimented with individual and combined augmentation methods and evaluated our framework on three public real-world datasets. Our results show that we were able to reduce the average MAE from 6.9 to 2.0.

Related papers

• SynFER: Towards Boosting Facial Expression Recognition with Synthetic Data [44.304022773272415]
  We introduce SynFER, a novel framework for synthesizing facial expression image data based on high-level textual descriptions. We propose a semantic guidance technique to steer the generation process and a pseudo-label generator to help rectify the facial expression labels. Our approach achieves a 67.23% classification accuracy on AffectNet when training solely with synthetic data equivalent to the AffectNet training set size.
  arXiv  Detail & Related papers  (2024-10-13T14:58:21Z)
• Towards Synthetic Data Generation for Improved Pain Recognition in Videos under Patient Constraints [11.515273901289472]
  This study introduces a novel approach that leverages synthetic data to enhance video-based pain recognition models. We present a pipeline that synthesizes realistic 3D facial models by capturing nuanced facial movements from a small participant pool. This process generates 8,600 synthetic faces, accurately reflecting genuine pain expressions from varied angles and perspectives.
  arXiv  Detail & Related papers  (2024-09-24T18:33:57Z)
• Synthetic Image Learning: Preserving Performance and Preventing Membership Inference Attacks [5.0243930429558885]
  This paper introduces Knowledge Recycling (KR), a pipeline designed to optimise the generation and use of synthetic data for training downstream classifiers. At the heart of this pipeline is Generative Knowledge Distillation (GKD), the proposed technique that significantly improves the quality and usefulness of the information. The results show a significant reduction in the performance gap between models trained on real and synthetic data, with models based on synthetic data outperforming those trained on real data in some cases.
  arXiv  Detail & Related papers  (2024-07-22T10:31:07Z)
• Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention [0.8192907805418583]
  We present two key risks of using synthetic data in model development. First, we detail the high risk of false confidence when using synthetic data to increase dataset diversity and representation. Second, we examine how using synthetic data risks circumventing consent for data usage.
  arXiv  Detail & Related papers  (2024-05-03T02:47:44Z)
• Best Practices and Lessons Learned on Synthetic Data [83.63271573197026]
  The success of AI models relies on the availability of large, diverse, and high-quality datasets. Synthetic data has emerged as a promising solution by generating artificial data that mimics real-world patterns.
  arXiv  Detail & Related papers  (2024-04-11T06:34:17Z)
• SDFR: Synthetic Data for Face Recognition Competition [51.9134406629509]
  Large-scale face recognition datasets are collected by crawling the Internet and without individuals' consent, raising legal, ethical, and privacy concerns. Recently several works proposed generating synthetic face recognition datasets to mitigate concerns in web-crawled face recognition datasets. This paper presents the summary of the Synthetic Data for Face Recognition (SDFR) Competition held in conjunction with the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2024) The SDFR competition was split into two tasks, allowing participants to train face recognition systems using new synthetic datasets and/or existing ones.
  arXiv  Detail & Related papers  (2024-04-06T10:30:31Z)
• Learning Human Action Recognition Representations Without Real Humans [66.61527869763819]
  We present a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model. We then evaluate the transferability of the representation learned on this data to a diverse set of downstream action recognition benchmarks. Our approach outperforms previous baselines by up to 5%.
  arXiv  Detail & Related papers  (2023-11-10T18:38:14Z)
• Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark [56.8042116967334]
  Synthetic data serves as an alternative in training machine learning models. ensuring that synthetic data mirrors the complex nuances of real-world data is a challenging task. This paper explores the potential of integrating data-centric AI techniques to guide the synthetic data generation process.
  arXiv  Detail & Related papers  (2023-10-25T20:32:02Z)
• ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning [102.46382882098847]
  We first investigate the effects of synthetic data in synthetic-to-real novel view synthesis. We propose to introduce geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints. Our method can render images with higher quality and better fine-grained details, outperforming existing generalizable novel view synthesis methods in terms of PSNR, SSIM, and LPIPS.
  arXiv  Detail & Related papers  (2023-03-20T12:06:14Z)
• Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances [76.34037366117234]
  We introduce a new dataset called Robot Control Gestures (RoCoG-v2) The dataset is composed of both real and synthetic videos from seven gesture classes. We present results using state-of-the-art action recognition and domain adaptation algorithms.
  arXiv  Detail & Related papers  (2023-03-17T23:23:55Z)
• Fake It Till You Make It: Face analysis in the wild using synthetic data alone [9.081019005437309]
  We show that it is possible to perform face-related computer vision in the wild using synthetic data alone. We describe how to combine a procedurally-generated 3D face model with a comprehensive library of hand-crafted assets to render training images with unprecedented realism.
  arXiv  Detail & Related papers  (2021-09-30T13:07:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.