Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography
- URL: http://arxiv.org/abs/2503.17351v1
- Date: Fri, 21 Mar 2025 17:52:33 GMT
- Title: Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography
- Authors: Vineet R. Shenoy, Shaoju Wu, Armand Comas, Tim K. Marks, Suhas Lohit, Hassan Mansour,
- Abstract summary: Photoplethysmography system consists of three modules: face and landmark detection, time-series extraction, and pulse signal/pulse rate estimation.<n>The pulse signal estimation module, which we call TURNIP, allows the system to faithfully reconstruct the underlying pulse signal waveform.<n>Our algorithm provides reliable heart rate estimates without the need for specialized sensors or contact with the skin.
- Score: 14.749406169315554
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Remote estimation of vital signs enables health monitoring for situations in which contact-based devices are either not available, too intrusive, or too expensive. In this paper, we present a modular, interpretable pipeline for pulse signal estimation from video of the face that achieves state-of-the-art results on publicly available datasets.Our imaging photoplethysmography (iPPG) system consists of three modules: face and landmark detection, time-series extraction, and pulse signal/pulse rate estimation. Unlike many deep learning methods that make use of a single black-box model that maps directly from input video to output signal or heart rate, our modular approach enables each of the three parts of the pipeline to be interpreted individually. The pulse signal estimation module, which we call TURNIP (Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography), allows the system to faithfully reconstruct the underlying pulse signal waveform and uses it to measure heart rate and pulse rate variability metrics, even in the presence of motion. When parts of the face are occluded due to extreme head poses, our system explicitly detects such "self-occluded" regions and maintains estimation robustness despite the missing information. Our algorithm provides reliable heart rate estimates without the need for specialized sensors or contact with the skin, outperforming previous iPPG methods on both color (RGB) and near-infrared (NIR) datasets.
Related papers
- EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance [79.66329903007869]
We present EchoWorld, a motion-aware world modeling framework for probe guidance.
It encodes anatomical knowledge and motion-induced visual dynamics.
It is trained on more than one million ultrasound images from over 200 routine scans.
arXiv Detail & Related papers (2025-04-17T16:19:05Z) - Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models [45.94962431110573]
Camera-based monitoring of vital signs, also known as imaging photoplethysmography (i), has seen applications in driver-monitoring, affective computing, and more.<n>We introduce methods that combine signal processing and deep learning methods in an inverse problem.
arXiv Detail & Related papers (2025-03-21T16:11:21Z) - SiNC+: Adaptive Camera-Based Vitals with Unsupervised Learning of Periodic Signals [6.458510829614774]
We present the first non-contrastive unsupervised learning framework for signal regression.
We find that encouraging sparse power spectra within normal physiological bandlimits and variance over batches of power spectra is sufficient for learning periodic signals.
arXiv Detail & Related papers (2024-04-20T19:17:40Z) - DopUS-Net: Quality-Aware Robotic Ultrasound Imaging based on Doppler
Signal [48.97719097435527]
DopUS-Net combines the Doppler images with B-mode images to increase the segmentation accuracy and robustness of small blood vessels.
An artery re-identification module qualitatively evaluate the real-time segmentation results and automatically optimize the probe pose for enhanced Doppler images.
arXiv Detail & Related papers (2023-05-15T18:19:29Z) - Non-Contrastive Unsupervised Learning of Physiological Signals from
Video [4.8327232174895745]
We present the first non-contrastive unsupervised learning framework for signal regression to break free from labelled video data.
With minimal assumptions of periodicity and finite bandwidth, our approach is capable of discovering blood volume pulse directly from unlabelled videos.
arXiv Detail & Related papers (2023-03-14T14:34:51Z) - Dataset Creation Pipeline for Camera-Based Heart Rate Estimation [0.3058685580689604]
Heart rate is one of the most vital health metrics which can be utilized to investigate and gain intuitions into various human physiological and psychological information.
Various techniques for camera-based heart rate estimation have been developed ranging from classical image processing to convoluted deep learning models and architectures.
In this paper, we discuss how to prepare data for the task of developing an algorithm or machine learning model for heart rate estimation from images of facial regions.
arXiv Detail & Related papers (2023-03-02T18:28:29Z) - WPPG Net: A Non-contact Video Based Heart Rate Extraction Network
Framework with Compatible Training Capability [21.33542693986985]
Our facial skin presents subtle color change known as remote Photoplethys (r) signal, from which we could extract the heart rate of the subject.
Recently many deep learning methods and related datasets on r signal extraction are proposed.
However, because of the time consumption blood flowing through our body and other factors, label waves such as BVP signals have uncertain delays with real r signals in some datasets.
In this paper, by analyzing the common characteristics on rhythm and periodicity of r signals and label waves, we propose a whole set of training methodology which wraps these networks so that they could remain efficient when be trained at
arXiv Detail & Related papers (2022-07-04T19:52:30Z) - Identifying Rhythmic Patterns for Face Forgery Detection and
Categorization [46.21354355137544]
We propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals.
With insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework.
arXiv Detail & Related papers (2022-07-04T04:57:06Z) - Tattoo tomography: Freehand 3D photoacoustic image reconstruction with
an optical pattern [49.240017254888336]
Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties.
A current drawback is the limited field-of-view provided by the conventionally applied 2D probes.
We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
arXiv Detail & Related papers (2020-11-10T09:27:56Z) - Video-based Remote Physiological Measurement via Cross-verified Feature
Disentangling [121.50704279659253]
We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations.
We then use the distilled physiological features for robust multi-task physiological measurements.
The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
arXiv Detail & Related papers (2020-07-16T09:39:17Z) - AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement
with Neural Searching [76.4844593082362]
We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS)
Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
arXiv Detail & Related papers (2020-04-26T05:43:21Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.