CardiacMamba: Fair and Robust RGB-RF Fusion for Remote Heart Rate Estimation via State Space Modeling

• URL: http://arxiv.org/abs/2608.15831v1
• Date: Sun, 16 Aug 2026 16:05:24 GMT
• Title: CardiacMamba: Fair and Robust RGB-RF Fusion for Remote Heart Rate Estimation via State Space Modeling
• Abstract summary: CardiacMamba integrates optical facial cues and radio-frequency cardiac motion cues through state space modeling.<n>On the EquiPleth dataset, CardiacMamba achieves state-of-the-art performance with 0.96 bpm MAE, 3.06 bpm RMSE, and 0.97 Pearson correlation.
• Score: 36.18750971669015
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Remote photoplethysmography (rPPG) enables non-contact heart rate (HR) monitoring from facial videos, but RGB-only methods are vulnerable to illumination changes, motion artifacts, and skin-tone-dependent optical reflectance. We propose CardiacMamba, a fair and robust RGB-RF fusion framework that integrates optical facial cues and radio-frequency cardiac motion cues through state space modeling. CardiacMamba introduces a Temporal Difference Mamba Module (TDMM) to enhance subtle RF temporal variations, a bidirectional SSM-based interaction mechanism to align heterogeneous RGB-RF dynamics, and a Channel-wise Fast Fourier Transform (CFFT) module for channel-domain spectral refinement. On the EquiPleth dataset, CardiacMamba achieves state-of-the-art performance with 0.96 bpm MAE, 3.06 bpm RMSE, and 0.97 Pearson correlation, while reducing the observed light-dark skin-tone MAE gap to 0.26 bpm and maintaining robustness under RGB degradation and RF-missing conditions

Related papers

• RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement [25.785573372322137]
  Video-based remote physiological measurement (RPM) is highly accessible but remains fragile under varying illumination, skin tones, and motion.<n>We propose RPM-Distill, a physiology-guided cross-modal distillation framework that leverages synchronized radar only during training.<n> RPM-Distill brings 81% MAE and 21% correlation improvement over unimodal baselines.
  arXiv  Detail & Related papers  (2026-06-26T13:50:47Z)
• Frequency upconversion of infrared signals via molecular cavity optomechanical systems with gain [4.857867070917862]
  We propose a scheme that incorporates gain into the infrared cavity of a molecular cavity optomechanical system comprising two cavities and an ensemble of N molecules.<n>The upconversion process, which relies on IR absorption and Raman scattering associated with specific vibrational modes, is significantly amplified by the incorporation of gain under the red-detuned conditions.
  arXiv  Detail & Related papers  (2026-06-16T12:50:13Z)
• Fusion-E2Pulse: A Multimodal Event-RGB Fusion Network for Non-contact Pulse Wave Reconstruction [9.207646887958935]
  Non-contact pulse wave reconstruction hinges on the precise recovery of waveform morphology, including the dicrotic notch.<n>We propose a novel multimodal network named Fusion-E2Pulse to exploit the synergy between frame-based integration and event-based differential sensing.<n>Experiments demonstrate that Fusion-E2Pulse achieves state-of-the-art performance, effectively balancing noise suppression and morphological fidelity.
  arXiv  Detail & Related papers  (2026-06-14T04:48:27Z)
• Illumination-Robust Camera-Based Heart-Rate Estimation for Physiological Sensing in Robots [3.7597760118057884]
  This paper presents an end-to-end spatial-temporal transformer framework for remote HR estimation on a new dataset with varied illumination.<n>Our estimator integrates PRNet-based 3D face alignment, clip-level illumination augmentation, the Residual Temporal Standardization Module, and controlled hybrid temporal-frequency supervision.
  arXiv  Detail & Related papers  (2026-06-10T17:42:31Z)
• PhysNeXt: Next-Generation Dual-Branch Structured Attention Fusion Network for Remote Photoplethysmography Measurement [50.524262997433546]
  rmography enables measurement of heart rate and other vital signs by analyzing color variations in facial skin induced by cardiac pulsation.<n>Current methods are mainly based on either end-to-end modeling from raw videos or subtle heartbeat-related map (ST) representations.<n>We propose PhysMapXt, a dual-input deep learning framework that jointly exploits video frames and ST representations.
  arXiv  Detail & Related papers  (2026-03-20T08:37:02Z)
• PHASE-Net: Physics-Grounded Harmonic Attention System for Efficient Remote Photoplethysmography Measurement [63.007237197267834]
  Existing deep learning methods are mostly physiological monitoring and lack theoretical robustness.<n>We propose a physics-informed r paradigm derived from the Navier-Stokes equations of hemodynamics, showing that the pulse signal follows a second-order system.<n>This provides a theoretical justification for using a Temporal Conal Network (TCN)<n>Phase-Net achieves state-of-the-art performance with strong efficiency, offering a theoretically grounded and deployment-ready r solution.
  arXiv  Detail & Related papers  (2025-09-29T14:36:45Z)
• CardiacMamba: A Multimodal RGB-RF Fusion Framework with State Space Models for Remote Physiological Measurement [24.511384674989223]
  Heart rate (HR) estimation via remote photoplethys (rPl) offers a non-invasive solution for health monitoring.<n>Traditional single-modality approaches (RGB or Radio Frequency (RF)) face challenges in balancing robustness and accuracy due to lighting variations, motion artifacts, and skin tone bias.<n>We propose CardiacMamba, a multimodal RGB-RF fusion framework that leverages the complementary strengths of both modalities.
  arXiv  Detail & Related papers  (2025-02-19T11:00:34Z)
• Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution [54.293362972473595]
  Image super-resolution (SR) aims to reconstruct high-resolution (HR) images from their low-resolution (LR) counterparts. Current approaches to address SR tasks are either dedicated to extracting RGB image features or assuming similar degradation patterns. We propose a Contourlet refinement gate framework to restore infrared modal-specific features while preserving spectral distribution fidelity.
  arXiv  Detail & Related papers  (2024-11-19T14:24:03Z)
• Frequency Domain Modality-invariant Feature Learning for Visible-infrared Person Re-Identification [79.9402521412239]
  We propose a novel Frequency Domain modality-invariant feature learning framework (FDMNet) to reduce modality discrepancy from the frequency domain perspective. Our framework introduces two novel modules, namely the Instance-Adaptive Amplitude Filter (IAF) and the Phrase-Preserving Normalization (PPNorm)
  arXiv  Detail & Related papers  (2024-01-03T17:11:27Z)
• Diffusion Probabilistic Model Made Slim [128.2227518929644]
  We introduce a customized design for slim diffusion probabilistic models (DPM) for light-weight image synthesis. We achieve 8-18x computational complexity reduction as compared to the latent diffusion models on a series of conditional and unconditional image generation tasks.
  arXiv  Detail & Related papers  (2022-11-27T16:27:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.