Foundation Model for Cardiac Time Series via Masked Latent Attention

• URL: http://arxiv.org/abs/2603.26475v1
• Date: Fri, 27 Mar 2026 14:41:44 GMT
• Title: Foundation Model for Cardiac Time Series via Masked Latent Attention
• Abstract summary: Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis.<n>We introduce the latent attention masked autoencoder (LAMAE) FM that directly exploits this structure by learning cross-lead connection mechanisms during self-supervised pretraining.<n>Our approach models higher-order interactions across leads through latent attention, enabling permutation-invariant aggregation and adaptive weighting of lead-specific representations.
• Score: 19.265319330946465
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent foundation models (FMs) have shown promise for learning transferable ECG representations, most existing pretraining approaches treat leads as independent channels and fail to explicitly leverage their strong structural redundancy. We introduce the latent attention masked autoencoder (LAMAE) FM that directly exploits this structure by learning cross-lead connection mechanisms during self-supervised pretraining. Our approach models higher-order interactions across leads through latent attention, enabling permutation-invariant aggregation and adaptive weighting of lead-specific representations. We provide empirical evidence on the Mimic-IV-ECG database that leveraging the cross-lead connection constitutes an effective form of structural supervision, improving representation quality and transferability. Our method shows strong performance in predicting ICD-10 codes, outperforming independent-lead masked modeling and alignment-based baselines.

Related papers

• Physics-Informed Deep Learning for False Ventricular Tachycardia Alarm Reduction in the ICU [44.313512671851306]
  False ventricular tachycardia (VT) alarms are a leading contributor to alarm fatigue in intensive care units.<n>We propose a deep learning framework combining a 1D SE-ResNet with ICU-realistic data augmentations and a physics-informed auxiliary reconstruction task.
  arXiv  Detail & Related papers  (2026-09-08T16:33:07Z)
• ProCA: Progressive Contrastive Alignment for Robust EEG Visual Decoding [47.35612571473169]
  EEG visual decoding aims to recover visual semantics from non-invasive neural time-series signals.<n>Existing methods rely on fixed visual or textual anchors whose semantic relations may become misaligned with EEG representations that vary across trials, subjects, and learning stages.<n>We propose Progressive Contrastive Alignment (ProCA), a unified and model-agnostic framework for adaptive neural-semantic alignment.
  arXiv  Detail & Related papers  (2026-09-04T12:48:25Z)
• Graph-Based Pseudo-multimodal Contrastive Learning for 12-Lead ECG Representations [1.671051893883682]
  We propose a graph-based pseudo-multimodal contrastive learning framework called Graph-CMMC.<n>ECG waveforms are transformed into Gramian Angular Difference Field (GADF) images to construct complementary representations of the same cardiac activity.<n>Using all 12 leads, Graph-CMMC aligns waveform and GADF representations in a self-supervised manner, while a graph-based relational module is employed to model inter-lead dependency.
  arXiv  Detail & Related papers  (2026-08-27T11:02:13Z)
• Graph-Supervised Hierarchical Clinical Alignment for Radiology Report Generation with Large Language Models [46.202482780666635]
  Graph-Hierarchicald Supervised Clinical Alignment reformulates image-report supervision as a hierarchical clinical alignment problem.<n>Our method structures this alignment as a disease-conditioned process, where supervision is decomposed into two levels.<n> Experiments on MIMIC-CXR, IU-Xray, and COV-CTR show that our method consistently improves performance on both conventional and clinical metrics.
  arXiv  Detail & Related papers  (2026-08-25T06:32:26Z)
• Autoregressive EHR Foundation Models with Multimodal Inputs [14.402677557506367]
  Autoregressive foundation models trained on tokenized electronic health records can support zero-shot clinical prediction.<n>We present a framework for conditioning such models on auxiliary clinical modalities, including ECG waveforms, chest X-ray images, and clinical notes.
  arXiv  Detail & Related papers  (2026-07-24T12:59:20Z)
• Transferring Clinical Knowledge into ECGs Representation [0.19498378931702776]
  We propose a novel three-stage training paradigm that transfers knowledge from multimodal clinical data into a powerful, yet unimodal, ECG encoder.<n>We employ a self-supervised, joint-embedding pre-training stage to create an ECG representation that is enriched with contextual clinical information.<n>As an indirect way to explain the model's output, we train it to also predict associated laboratory abnormalities directly from the ECG embedding.
  arXiv  Detail & Related papers  (2025-12-07T22:19:24Z)
• Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models [45.63522160275318]
  Vision Foundation Model Variational Autoencoder (VFM-VAE) designed to resolve inherent tension between VFM's semantic focus and need for pixel-level fidelity.<n>Our system reaches a gFID (w/o CFG) of 2.20 in merely 80 epochs (a 10x speedup over prior tokenizers)
  arXiv  Detail & Related papers  (2025-10-21T09:30:45Z)
• Mitigating Attention Hacking in Preference-Based Reward Modeling via Interaction Distillation [62.14692332209628]
  "Interaction Distillation" is a novel training framework for more adequate preference modeling through attention-level optimization.<n>It provides more stable and generalizable reward signals compared to state-of-the-art RM optimization methods.
  arXiv  Detail & Related papers  (2025-08-04T17:06:23Z)
• Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals [36.08910150609342]
  We present a cardiac sensing foundation model (CSFM) that learns unified representations from vast, heterogeneous health records.<n>Our model is pretrained on an innovative multi-modal integration of data from multiple large-scale datasets.<n> CSFM consistently outperforms traditional one-modal-one-task approaches.
  arXiv  Detail & Related papers  (2025-06-23T20:58:12Z)
• Channel-Imposed Fusion: A Simple yet Effective Method for Medical Time Series Classification [18.061020420145407]
  This study shifts focus toward a modeling paradigm that emphasizes structural transparency, aligning more closely with the intrinsic characteristics of medical data.<n>We propose a novel method, Channel Imposed Fusion (CIF), which enhances the signal-to-noise ratio through cross-channel information fusion.<n> Experimental results on multiple publicly available EEG and ECG datasets demonstrate that the proposed method not only outperforms existing state-of-the-art (SOTA) approaches in terms of various classification metrics, but also significantly enhances the transparency of the classification process, offering a novel perspective for medical time series classification.
  arXiv  Detail & Related papers  (2025-05-31T01:44:30Z)
• Channel Fingerprint Construction for Massive MIMO: A Deep Conditional Generative Approach [65.47969413708344]
  We introduce the concept of CF twins and design a conditional generative diffusion model (CGDM)<n>We employ a variational inference technique to derive the evidence lower bound (ELBO) for the log-marginal distribution of the observed fine-grained CF conditioned on the coarse-grained CF.<n>We show that the proposed approach exhibits significant improvement in reconstruction performance compared to the baselines.
  arXiv  Detail & Related papers  (2025-05-12T01:36:06Z)
• Self-supervised inter-intra period-aware ECG representation learning for detecting atrial fibrillation [41.82319894067087]
  We propose an inter-intra period-aware ECG representation learning approach. Considering ECGs of atrial fibrillation patients exhibit the irregularity in RR intervals and the absence of P-waves, we develop specific pre-training tasks for interperiod and intraperiod representations. Our approach demonstrates remarkable AUC performances on the BTCH dataset, textiti.e., 0.953/0.996 for paroxysmal/persistent atrial fibrillation detection.
  arXiv  Detail & Related papers  (2024-10-08T10:03:52Z)
• Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners [10.088785685439134]
  We propose D-BETA, a framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture.<n>D-BETA uniquely combines the strengths of generative with boosted discriminative capabilities to achieve robust cross-modal representations.
  arXiv  Detail & Related papers  (2024-10-03T01:24:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.