Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning

• URL: http://arxiv.org/abs/2605.00973v1
• Date: Fri, 01 May 2026 17:04:15 GMT
• Title: Physiology-Aware Masked Cross-Modal Reconstruction for Biosignal Representation Learning
• Abstract summary: We introduce xMAE, a biosignal pretraining framework that leverages cross modal reconstruction across temporally ordered biosignals as a training time constraint.<n>We show that pretraining with xMAE yields representations that outperform both unimodal and multimodal baselines on 15 of 19 downstream tasks.<n>xMAE demonstrates the effectiveness of incorporating temporal structure into multimodal pretraining signals observe different stages of a shared underlying process.
• Score: 14.840342258743448
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biosignals acquired from different locations on the body often provide temporally ordered views of the same underlying physiological process. However, most existing self supervised learning methods treat these signals as interchangeable views, overlooking the directional temporal dynamics that link them. A canonical example is the relationship between electrocardiography (ECG), which captures the electrical activation initiating each heartbeat, and photoplethysmography (PPG), which records the resulting peripheral pulse delayed by vascular dynamics. To capture this structured relationship, we introduce xMAE, a biosignal pretraining framework that leverages masked cross modal reconstruction across temporally ordered biosignals as a training time constraint to encourage physiologically meaningful timing structure in the learned representations. We show that pretraining with xMAE yields representations that outperform both unimodal and multimodal baselines on 15 of 19 downstream tasks, including cardiovascular outcome prediction, abnormal laboratory test detection, sleep staging, and demographic inference, while generalizing across devices, body locations, and acquisition settings. Further analysis suggests that the ECG PPG timing structure is reflected in the learned PPG representations. More broadly, xMAE demonstrates the effectiveness of incorporating temporal structure into multimodal pretraining when signals observe different stages of a shared underlying process. Code is available at https://github.com/hzhou3/xMAE.

Related papers

• How Good are Foundation Models in Longitudinal MRI Disease Progression Reasoning? [49.14050864978079]
  We introduce the Time-Aware Multi-View MRI Benchmark, an evaluation framework unifying multi-view anatomical input, temporal reasoning, and structured localization guidance.<n>The benchmark comprises 3,920 expert-verified question-answer pairs derived from 890 patients across over 3,200 longitudinal MRI timepoints.
  arXiv  Detail & Related papers  (2026-08-13T14:35:20Z)
• The Impact of Temporal Context Length and Encoding Strategies on Self-Supervised ECG Representation Learning [0.8411385346896411]
  Self-supervised electrocardiogram (ECG) models are often trained on a few seconds of ECG signal.<n>It remains unclear whether these choices sacrifice information needed for rhythm inference and longitudinal consistency.<n>We present a controlled study on the Icentia11k single-lead dataset.
  arXiv  Detail & Related papers  (2026-08-13T01:18:39Z)
• Rhythm-Structured Predictive Learning for Remote Photoplethysmography [0.0]
  RhythmJEPA predicts latent teacher representations from masked facial videos.<n>CRSP estimates frame-wise latent physiological states and decodes the most plausible cyclic state path.<n>DOM captures both local temporal continuity and long-range rhythm-consistent dependencies.<n>SPM extracts compact pulse-sensitive facial tokens with a favorable balance between complexity and performance.
  arXiv  Detail & Related papers  (2026-06-30T14:32:53Z)
• Reperio-rPPG: Relational Temporal Graph Neural Networks for Periodicity Learning in Remote Physiological Measurement [0.0]
  Remote photoplethys (rmography) is an emerging physiological sensing technique that leverages subtle color variations in facial videos to estimate vital signs such as heart rate and respiratory rate.<n>This non-invasive technique has gained traction across diverse domains, but its ability to capture fine-grained temporal dynamics under real-world conditions has been underexplored.<n>We propose Graph Reperio-r, a novel framework that strategically integrates a Transformer to effectively capture the periodic structure.
  arXiv  Detail & Related papers  (2025-11-08T09:41:34Z)
• S4ECG: Exploring the impact of long-range interactions for arrhythmia prediction [1.6873748786804317]
  We introduce S4ECG, a novel deep learning architecture leveraging structured state space models for multi-epoch arrhythmia classification.<n>Our joint multi-epoch predictions significantly outperform single-epoch approaches by 1.0-11.6% in macro-AUROC.<n>This work contributes to a paradigm shift toward temporally-aware arrhythmia detection algorithms, opening new possibilities for ECG interpretation.
  arXiv  Detail & Related papers  (2025-10-20T10:48:44Z)
• Masked Autoencoders that Feel the Heart: Unveiling Simplicity Bias for ECG Analyses [24.039917512972977]
  We show that ECG models tend to overfit dominant and repetitive patterns, overlooking fine-grained but clinically critical cues, known as Simplicity Bias (SB)<n>We propose a novel method comprising two key components: 1) Temporal-frequency aware Filters to capture temporal-frequency features reflecting the dynamic characteristics of ECG signals, and 2) building on this, Multi-Grained Prototype Reconstruction for coarse and fine representation learning across dual domains, further mitigating SB.
  arXiv  Detail & Related papers  (2025-06-25T03:25:49Z)
• Cross-Learning Between ECG and PCG: Exploring Common and Exclusive Characteristics of Bimodal Electromechanical Cardiac Waveforms [6.81602773853736]
  We investigate the common and exclusive characteristics of ECG and PCG.<n>We employ a suite of linear and nonlinear machine learning models to reconstruct each modality from the other.<n> clinically relevant ECG biomarkers, such as fiducial points and QT intervals, can be estimated from PCG in cross-subject settings.
  arXiv  Detail & Related papers  (2025-06-11T22:16:59Z)
• PhysLLM: Harnessing Large Language Models for Cross-Modal Remote Physiological Sensing [49.243031514520794]
  Large Language Models (LLMs) excel at capturing long-range signals due to their text-centric design.<n>PhysLLM achieves state-the-art accuracy and robustness, demonstrating superior generalization across lighting variations and motion scenarios.
  arXiv  Detail & Related papers  (2025-05-06T15:18:38Z)
• PSDNorm: Test-Time Temporal Normalization for Deep Learning in Sleep Staging [63.05435596565677]
  We propose PSDNorm that leverages Monge mapping and temporal context to normalize feature maps in deep learning models for signals.<n> PSDNorm achieves state-of-the-art performance on unseen left-out datasets while being 4-times more data-efficient than BatchNorm.
  arXiv  Detail & Related papers  (2025-03-06T16:20:25Z)
• CTPD: Cross-Modal Temporal Pattern Discovery for Enhanced Multimodal Electronic Health Records Analysis [50.56875995511431]
  We introduce a Cross-Modal Temporal Pattern Discovery (CTPD) framework, designed to efficiently extract meaningful cross-modal temporal patterns from multimodal EHR data.<n>Our approach introduces shared initial temporal pattern representations which are refined using slot attention to generate temporal semantic embeddings.
  arXiv  Detail & Related papers  (2024-11-01T15:54:07Z)
• PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer [76.40106756572644]
  Recent deep learning approaches focus on mining subtle clues using convolutional neural networks with limited-temporal receptive fields. In this paper, we propose two end-to-end video transformer based on PhysFormer and Phys++++, to adaptively aggregate both local and global features for r representation enhancement. Comprehensive experiments are performed on four benchmark datasets to show our superior performance on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2023-02-07T15:56:03Z)
• Learning ECG Representations based on Manipulated Temporal-Spatial Reverse Detection [11.615287369669971]
  We propose a straightforward but effective approach to learn ECG representations. Inspired by the temporal and spatial characteristics of ECG, we flip the original signals horizontally, vertically, and both horizontally and vertically. Results show that the ECG representations learned with our method lead to remarkable performances on the downstream task.
  arXiv  Detail & Related papers  (2022-02-25T02:01:09Z)
• PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer [55.936527926778695]
  Recent deep learning approaches focus on mining subtle r clues using convolutional neural networks with limited-temporal receptive fields. In this paper, we propose the PhysFormer, an end-to-end video transformer based architecture.
  arXiv  Detail & Related papers  (2021-11-23T18:57:11Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.