Illumination-Robust Camera-Based Heart-Rate Estimation for Physiological Sensing in Robots

• URL: http://arxiv.org/abs/2606.12378v1
• Date: Wed, 10 Jun 2026 17:42:31 GMT
• Title: Illumination-Robust Camera-Based Heart-Rate Estimation for Physiological Sensing in Robots
• Abstract summary: This paper presents an end-to-end spatial-temporal transformer framework for remote HR estimation on a new dataset with varied illumination.<n>Our estimator integrates PRNet-based 3D face alignment, clip-level illumination augmentation, the Residual Temporal Standardization Module, and controlled hybrid temporal-frequency supervision.
• Score: 3.7597760118057884
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Physiological awareness is important for service, social, and assistive robots that interact with humans in everyday environments. Remote photoplethysmography (rPPG) enables non-contact heart-rate (HR) estimation from an RGB camera, making it a promising sensing modality for robot-mounted vision systems. However, illumination variation remains a major barrier to robust deployment. This paper presents an end-to-end spatial-temporal transformer framework for remote HR estimation on a new dataset with varied illumination. Our estimator integrates PRNet-based 3D face alignment, clip-level illumination augmentation, the Residual Temporal Standardization Module, and controlled hybrid temporal-frequency supervision. The training objective combines a Soft-Shifted Pearson waveform loss with a spectral Kullback-Leibler divergence loss, where a tuned weight ($\mathbfβ$) controls the contribution of frequency-domain heart-rate guidance. Experiments on a static all-level mix protocol covering three illumination levels show that $\mathbfβ=5$ provides the strongest result among the tested beta settings, achieving a best-run HR mean absolute error (MAE) of 0.79 bpm and an HR correlation of 0.982. Compared with the PhysFormer baseline evaluated on our dataset, our estimator reduces HR MAE by 93.6 %, while increasing HR correlation from 0.088 to 0.982, making it usable when illumination varies.

Related papers

• Not All Relations Are Equal: Relation-Balanced and Calibrated Graph Learning for Provenance-Based Intrusion Detection [66.44527094471619]
  We present RECAL, an unsupervised framework using relation-balanced masked graph learning to better capture rare interaction patterns.<n>On three DARPA E3 datasets, RECAL achieves F1 scores of 99.99%, 99.93%, and 99.99%, outperforming the best baseline on each dataset by 0.88, 0.82, and 0.42 percentage points, respectively.
  arXiv  Detail & Related papers  (2026-09-15T00:31:01Z)
• CardiacMamba: Fair and Robust RGB-RF Fusion for Remote Heart Rate Estimation via State Space Modeling [36.18750971669015]
  CardiacMamba integrates optical facial cues and radio-frequency cardiac motion cues through state space modeling.<n>On the EquiPleth dataset, CardiacMamba achieves state-of-the-art performance with 0.96 bpm MAE, 3.06 bpm RMSE, and 0.97 Pearson correlation.
  arXiv  Detail & Related papers  (2026-08-16T16:05:24Z)
• Factorized Spatio-Temporal Convolutions for Human Pose Estimation from Planar Lidar [43.576051446645856]
  Localizing nearby humans and estimating their facing direction are key capabilities for safe navigation and socially aware human-robot interaction.<n>We address omnidirectional human detection and relative 2D pose estimation from planar LiDAR sequences with a lightweight network based on Space-Time Blocks.<n>Our network processes 360 LiDAR sequences to output per-ray human presence, distance, and relative orientation.
  arXiv  Detail & Related papers  (2026-07-23T13:31:54Z)
• Amplifying, Not Learning: Fine-Tuned AI Text Detectors Amplify a Pretrained Direction [51.56484100374058]
  Text detectors amplify a pretrained typicality axis; they do not construct an AI-vs-human boundary.<n>On raw encoders before any task supervision, projecting onto centroid(AI)-centroid(HC3) achieves NYT-vs-HC3 AUROC 0.806/0.944/0.834 across three architectures.<n>On RoBERTa-base, raw projection exceeds fine-tuning; on RoBERTa-base, full fine-tuning reduces discrimination below raw on both fluent-formal populations tested.
  arXiv  Detail & Related papers  (2026-05-20T19:08:38Z)
• Triple Spectral Fusion for Sensor-based Human Activity Recognition [86.21994396715074]
  This paper proposes a novel triple spectral fusion framework tailored for sensor-based human activity recognition (HAR)<n>We develop an adaptive complementary filtering technique for noise suppression and organize each IMU's sensors into posture and motion modality nodes.<n>An adaptive wavelet frequency selection approach is implemented to suppress context redundancy and shorten the length of features.
  arXiv  Detail & Related papers  (2026-05-04T15:42:58Z)
• Thermal Imaging for Contactless Cardiorespiratory and Sudomotor Response Monitoring [9.779484408479108]
  This paper characterizes the extraction of electrodermal activity (EDA), heart rate (HR), and breathing rate (BR) signals from thermal video.<n>We evaluate 288 EDA configurations and the HR/BR pipeline on 31 sessions from the public SIMULATOR STUDY 1 driver monitoring dataset.
  arXiv  Detail & Related papers  (2026-02-12T19:41:18Z)
• CardiacMamba: A Multimodal RGB-RF Fusion Framework with State Space Models for Remote Physiological Measurement [24.511384674989223]
  Heart rate (HR) estimation via remote photoplethys (rPl) offers a non-invasive solution for health monitoring.<n>Traditional single-modality approaches (RGB or Radio Frequency (RF)) face challenges in balancing robustness and accuracy due to lighting variations, motion artifacts, and skin tone bias.<n>We propose CardiacMamba, a multimodal RGB-RF fusion framework that leverages the complementary strengths of both modalities.
  arXiv  Detail & Related papers  (2025-02-19T11:00:34Z)
• RRWaveNet: A Compact End-to-End Multi-Scale Residual CNN for Robust PPG Respiratory Rate Estimation [0.6464087844700315]
  Respiratory rate (RR) is an important biomarker as RR changes can reflect severe medical events such as heart disease, lung disease, and sleep disorders. Standard RR counting is prone to human error and cannot be performed continuously. This study proposes a method for continuously estimating RR, RRWaveNet.
  arXiv  Detail & Related papers  (2022-08-18T07:11:34Z)
• MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Video [75.23812405203778]
  Recent solutions have been introduced to estimate 3D human pose from 2D keypoint sequence by considering body joints among all frames globally to learn-temporal correlation. We propose Mix Mix, which has temporal transformer block to separately model the temporal motion of each joint and a transformer block inter-joint spatial correlation. In addition, the network output is extended from the central frame to entire frames of input video, improving the coherence between the input and output benchmarks.
  arXiv  Detail & Related papers  (2022-03-02T04:20:59Z)
• Embedding Temporal Convolutional Networks for Energy-Efficient PPG-Based Heart Rate Monitoring [17.155316991045765]
  Photoplethysmography (volution) sensors allow for non-invasive and comfortable heart-rate (HR) monitoring. Motion Artifacts (MAs) severely impact the monitoring accuracy, causing high variability in the skin-to-sensor interface. We propose a computationally lightweight yet robust deep learning-based approach for PPG-based HR estimation. We validate our approaches on two benchmark datasets, achieving as low as 3.84 Beats per Minute (BPM) of Mean Absolute Error (MAE) on PPGDalia.
  arXiv  Detail & Related papers  (2022-03-01T17:04:28Z)
• Online Body Schema Adaptation through Cost-Sensitive Active Learning [63.84207660737483]
  The work was implemented in a simulation environment, using the 7DoF arm of the iCub robot simulator. A cost-sensitive active learning approach is used to select optimal joint configurations. The results show cost-sensitive active learning has similar accuracy to the standard active learning approach, while reducing in about half the executed movement.
  arXiv  Detail & Related papers  (2021-01-26T16:01:02Z)
• MOMBAT: Heart Rate Monitoring from Face Video using Pulse Modeling and Bayesian Tracking [10.43230025523549]
  We propose a novel face video based HR monitoring method MOMBAT. We utilize out-of-plane face movements to define a novel quality estimation mechanism. We design a Bayesian decision theory based HR tracking mechanism to rectify the spurious HR estimates.
  arXiv  Detail & Related papers  (2020-05-10T09:41:16Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.