Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography

• URL: http://arxiv.org/abs/2503.17351v1
• Date: Fri, 21 Mar 2025 17:52:33 GMT
• Title: Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography
• Abstract summary: Photoplethysmography system consists of three modules: face and landmark detection, time-series extraction, and pulse signal/pulse rate estimation.<n>The pulse signal estimation module, which we call TURNIP, allows the system to faithfully reconstruct the underlying pulse signal waveform.<n>Our algorithm provides reliable heart rate estimates without the need for specialized sensors or contact with the skin.
• Score: 14.749406169315554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Remote estimation of vital signs enables health monitoring for situations in which contact-based devices are either not available, too intrusive, or too expensive. In this paper, we present a modular, interpretable pipeline for pulse signal estimation from video of the face that achieves state-of-the-art results on publicly available datasets.Our imaging photoplethysmography (iPPG) system consists of three modules: face and landmark detection, time-series extraction, and pulse signal/pulse rate estimation. Unlike many deep learning methods that make use of a single black-box model that maps directly from input video to output signal or heart rate, our modular approach enables each of the three parts of the pipeline to be interpreted individually. The pulse signal estimation module, which we call TURNIP (Time-Series U-Net with Recurrence for Noise-Robust Imaging Photoplethysmography), allows the system to faithfully reconstruct the underlying pulse signal waveform and uses it to measure heart rate and pulse rate variability metrics, even in the presence of motion. When parts of the face are occluded due to extreme head poses, our system explicitly detects such "self-occluded" regions and maintains estimation robustness despite the missing information. Our algorithm provides reliable heart rate estimates without the need for specialized sensors or contact with the skin, outperforming previous iPPG methods on both color (RGB) and near-infrared (NIR) datasets.

Related papers

• Non-contact, Real-time, Heart-rate Measurement using Image Processing with Commodity Cameras and AI Agents [1.1458853556386797]
  We develop a system for non-contact, real-time, heart rate measurement using image processing with commodity cameras such as an embedded camera on a laptop.<n>We use an innovative algorithm to capture the relevant signals for the computation of heart rate in a time series in real life environments.<n>We test and analyze the developed prototypes against heart rate results by Apple Watch.
  arXiv  Detail & Related papers  (2026-07-06T23:38:02Z)
• MS-rPPG: Multi-spectral State Space Model for Remote Photoplethysmography in Driver Monitoring Systems [21.37687518779808]
  In this study, we investigate a driver health monitoring system based on remote heart rate estimation.<n>We introduce MS-r, a multi-spectral framework that combines RGB with near-infrared (NIR) face video to alleviate r estimation under challenging driving conditions.<n>We collected a real-world dataset called MS-Drive, which was recorded from 50 participants while driving the vehicle.
  arXiv  Detail & Related papers  (2026-06-19T05:34:10Z)
• PhysNeXt: Next-Generation Dual-Branch Structured Attention Fusion Network for Remote Photoplethysmography Measurement [50.524262997433546]
  rmography enables measurement of heart rate and other vital signs by analyzing color variations in facial skin induced by cardiac pulsation.<n>Current methods are mainly based on either end-to-end modeling from raw videos or subtle heartbeat-related map (ST) representations.<n>We propose PhysMapXt, a dual-input deep learning framework that jointly exploits video frames and ST representations.
  arXiv  Detail & Related papers  (2026-03-20T08:37:02Z)
• SCG With Your Phone: Diagnosis of Rhythmic Spectrum Disorders in Field Conditions [0.0]
  We present a robust deep-learning framework for SCG segmentation and rhythm analysis using accelerometer recordings obtained with consumer smartphones.<n>We develop an enhanced U-Net v3 architecture that integrates multi-scale convolutions, residual connections, and attention gates.<n>A dedicated post-processing pipeline converts probability masks into precise AO timestamps, whereas a novel adaptive 3D-to-1D projection method ensures robustness to arbitrary smartphone orientation.
  arXiv  Detail & Related papers  (2026-01-20T12:58:31Z)
• 3D Blood Pulsation Maps [18.225199436812588]
  We present Pulse3DFace, the first dataset of its kind for estimating 3D blood pulsation maps.<n>These maps can be used to develop models of dynamic facial blood pulsation.<n>We offer a comprehensive evaluation of the dataset's illumination conditions, map consistency, and its ability to capture physiologically meaningful features in the facial and neck skin regions.
  arXiv  Detail & Related papers  (2025-12-11T10:40:31Z)
• PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement [63.007237197267834]
  Existing deep learning methods are mostly physiological monitoring and lack theoretical robustness.<n>We propose a physics-informed r paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order system.<n>This provides a theoretical justification for using a Temporal Conal Network (TCN)<n>Phase-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready r solution.
  arXiv  Detail & Related papers  (2025-09-29T14:36:45Z)
• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance [79.66329903007869]
  We present EchoWorld, a motion-aware world modeling framework for probe guidance. It encodes anatomical knowledge and motion-induced visual dynamics. It is trained on more than one million ultrasound images from over 200 routine scans.
  arXiv  Detail & Related papers  (2025-04-17T16:19:05Z)
• Recovering Pulse Waves from Video Using Deep Unrolling and Deep Equilibrium Models [45.94962431110573]
  Camera-based monitoring of vital signs, also known as imaging photoplethysmography (i), has seen applications in driver-monitoring, affective computing, and more.<n>We introduce methods that combine signal processing and deep learning methods in an inverse problem.
  arXiv  Detail & Related papers  (2025-03-21T16:11:21Z)
• SiNC+: Adaptive Camera-Based Vitals with Unsupervised Learning of Periodic Signals [6.458510829614774]
  We present the first non-contrastive unsupervised learning framework for signal regression. We find that encouraging sparse power spectra within normal physiological bandlimits and variance over batches of power spectra is sufficient for learning periodic signals.
  arXiv  Detail & Related papers  (2024-04-20T19:17:40Z)
• DopUS-Net: Quality-Aware Robotic Ultrasound Imaging based on Doppler Signal [48.97719097435527]
  DopUS-Net combines the Doppler images with B-mode images to increase the segmentation accuracy and robustness of small blood vessels. An artery re-identification module qualitatively evaluate the real-time segmentation results and automatically optimize the probe pose for enhanced Doppler images.
  arXiv  Detail & Related papers  (2023-05-15T18:19:29Z)
• Non-Contrastive Unsupervised Learning of Physiological Signals from Video [4.8327232174895745]
  We present the first non-contrastive unsupervised learning framework for signal regression to break free from labelled video data. With minimal assumptions of periodicity and finite bandwidth, our approach is capable of discovering blood volume pulse directly from unlabelled videos.
  arXiv  Detail & Related papers  (2023-03-14T14:34:51Z)
• Dataset Creation Pipeline for Camera-Based Heart Rate Estimation [0.3058685580689604]
  Heart rate is one of the most vital health metrics which can be utilized to investigate and gain intuitions into various human physiological and psychological information. Various techniques for camera-based heart rate estimation have been developed ranging from classical image processing to convoluted deep learning models and architectures. In this paper, we discuss how to prepare data for the task of developing an algorithm or machine learning model for heart rate estimation from images of facial regions.
  arXiv  Detail & Related papers  (2023-03-02T18:28:29Z)
• WPPG Net: A Non-contact Video Based Heart Rate Extraction Network Framework with Compatible Training Capability [21.33542693986985]
  Our facial skin presents subtle color change known as remote Photoplethys (r) signal, from which we could extract the heart rate of the subject. Recently many deep learning methods and related datasets on r signal extraction are proposed. However, because of the time consumption blood flowing through our body and other factors, label waves such as BVP signals have uncertain delays with real r signals in some datasets. In this paper, by analyzing the common characteristics on rhythm and periodicity of r signals and label waves, we propose a whole set of training methodology which wraps these networks so that they could remain efficient when be trained at
  arXiv  Detail & Related papers  (2022-07-04T19:52:30Z)
• Identifying Rhythmic Patterns for Face Forgery Detection and Categorization [46.21354355137544]
  We propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals. With insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework.
  arXiv  Detail & Related papers  (2022-07-04T04:57:06Z)
• Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern [49.240017254888336]
  Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties. A current drawback is the limited field-of-view provided by the conventionally applied 2D probes. We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
  arXiv  Detail & Related papers  (2020-11-10T09:27:56Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.