CREMA: A Contrastive Regularized Masked Autoencoder for Robust ECG Diagnostics across Clinical Domains

• URL: http://arxiv.org/abs/2407.07110v3
• Date: Thu, 21 Aug 2025 13:24:18 GMT
• Title: CREMA: A Contrastive Regularized Masked Autoencoder for Robust ECG Diagnostics across Clinical Domains
• Authors: Junho Song, Jong-Hwan Jang, DongGyun Hong, Joon-myoung Kwon, Yong-Yeon Jo,
• Abstract summary: We present CREMA, a foundation model for 12-lead ECGs designed to learn generalizable representations through self-supervised pretraining.<n> CREMA combines generative learning and contrastive regularization via a Contrastive Regularized MAE loss, and employs a Signal Transformer (SiT) architecture to capture both local waveform details and global temporal dependencies.
• Score: 2.9143698739149615
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Electrocardiogram (ECG) diagnosis remains challenging due to limited labeled data and the need to capture subtle yet clinically meaningful variations in rhythm and morphology. We present CREMA (Contrastive Regularized Masked Autoencoder), a foundation model for 12-lead ECGs designed to learn generalizable representations through self-supervised pretraining. CREMA combines generative learning and contrastive regularization via a Contrastive Regularized MAE loss, and employs a Signal Transformer (SiT) architecture to capture both local waveform details and global temporal dependencies. We evaluate CREMA on benchmark datasets and real-world clinical environments, including deployment scenarios with significant distribution shifts. CREMA outperforms supervised baselines and existing self-supervised models in both linear probing and fine-tuning evaluations. Notably, it maintains superior performance across diverse clinical domains, such as emergency care, highlighting its robustness under real-world conditions. These results demonstrate that CREMA serves as a scalable and reliable foundation model for ECG diagnostics, supporting downstream applications across heterogeneous and high-risk clinical settings.

Related papers

• ECG-RAMBA: Zero-Shot ECG Generalization by Morphology-Rhythm Disentanglement and Long-Range Modeling [0.6875312133832079]
  ECG-RAMBA is a framework that separates morphology and rhythm and re-integrates them through context-aware fusion.<n>It shows consistent cross-dataset performance on PTB-XL.
  arXiv  Detail & Related papers  (2025-12-29T10:14:52Z)
• Transferring Clinical Knowledge into ECGs Representation [0.19498378931702776]
  We propose a novel three-stage training paradigm that transfers knowledge from multimodal clinical data into a powerful, yet unimodal, ECG encoder.<n>We employ a self-supervised, joint-embedding pre-training stage to create an ECG representation that is enriched with contextual clinical information.<n>As an indirect way to explain the model's output, we train it to also predict associated laboratory abnormalities directly from the ECG embedding.
  arXiv  Detail & Related papers  (2025-12-07T22:19:24Z)
• Simulator and Experience Enhanced Diffusion Model for Comprehensive ECG Generation [52.19347532840774]
  We propose SE-Diff, a novel physiological simulator and experience enhanced diffusion model for ECG generation.<n> SE-Diff integrates a lightweight ordinary differential equation (ODE)-based ECG simulator into the diffusion process via a beat decoder.<n>Extensive experiments on real-world ECG datasets demonstrate that SE-Diff improves both signal fidelity and text-ECG semantic alignment.
  arXiv  Detail & Related papers  (2025-11-13T02:57:10Z)
• High-Fidelity Synthetic ECG Generation via Mel-Spectrogram Informed Diffusion Training [3.864395218585964]
  Development of machine learning for cardiac care is hampered by privacy restrictions on sharing real patient electrocardiogram (ECG) data.<n>In this work, we address two major shortcomings of current generative ECG methods.<n>We build on a conditional diffusion-based Structured State Space Model (SSSD-ECG) with two principled innovations.
  arXiv  Detail & Related papers  (2025-10-07T01:14:53Z)
• EEG-MedRAG: Enhancing EEG-based Clinical Decision-Making via Hierarchical Hypergraph Retrieval-Augmented Generation [45.031633614714]
  EEG-MedRAG is a three-layer hypergraph-based retrieval-augmented generation framework.<n>It unifies EEG domain knowledge, individual patient cases, and a large-scale repository into a traversable n-ary relational hypergraph.<n>We introduce the first cross-disease, cross-role EEG clinical QA benchmark, spanning seven disorders and five authentic clinical perspectives.
  arXiv  Detail & Related papers  (2025-08-19T11:12:58Z)
• Improving Myocardial Infarction Detection via Synthetic ECG Pretraining [0.0]
  Myocardial infarction is a major cause of death globally, and accurate early diagnosis from electrocardiograms (ECGs) remains a clinical priority.<n>Deep learning models have shown promise for automated ECG interpretation, but require large amounts of labeled data.<n>We propose a physiology-aware pipeline that synthesizes 12-lead ECGs with tunable MI morphology and realistic noise.
  arXiv  Detail & Related papers  (2025-06-29T14:29:55Z)
• Sensing Cardiac Health Across Scenarios and Devices: A Multi-Modal Foundation Model Pretrained on Heterogeneous Data from 1.7 Million Individuals [36.08910150609342]
  We present a cardiac sensing foundation model (CSFM) that learns unified representations from vast, heterogeneous health records.<n>Our model is pretrained on an innovative multi-modal integration of data from multiple large-scale datasets.<n> CSFM consistently outperforms traditional one-modal-one-task approaches.
  arXiv  Detail & Related papers  (2025-06-23T20:58:12Z)
• Heartcare Suite: Multi-dimensional Understanding of ECG with Raw Multi-lead Signal Modeling [50.58126509704037]
  Heartcare Suite is a framework for fine-grained electrocardiogram (ECG) understanding.<n>Heartcare-220K is a high-quality, structured, and comprehensive multimodal ECG dataset.<n>Heartcare-Bench is a benchmark to guide the optimization of Medical Multimodal Large Language Models (Med-MLLMs) in ECG scenarios.
  arXiv  Detail & Related papers  (2025-06-06T07:56:41Z)
• A CNN-based Local-Global Self-Attention via Averaged Window Embeddings for Hierarchical ECG Analysis [1.0844302367985357]
  We propose a novel Local-Global Attention ECG model (LGA-ECG) to address this limitation.<n>Our approach extracts queries by averaging embeddings obtained from overlapping convolutional windows.<n> Experiments conducted on the CODE-15 dataset demonstrate that LGA-ECG outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2025-04-13T01:21:18Z)
• GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images [44.50428701650495]
  We introduce GEM, the first MLLM unifying ECG time series, 12-lead ECG images and text for grounded and clinician-aligned ECG interpretation.<n> GEM enables feature-grounded analysis, evidence-driven reasoning, and a clinician-like diagnostic process through three core innovations.<n>We propose the Grounded ECG task, a clinically motivated benchmark designed to assess the MLLM's capability in grounded ECG understanding.
  arXiv  Detail & Related papers  (2025-03-08T05:48:53Z)
• Fusion of ECG Foundation Model Embeddings to Improve Early Detection of Acute Coronary Syndromes [5.723893680574976]
  This study explores the use of ECG foundation models, specifically ST-MEM and ECG-FM, to enhance ACS risk assessment. We evaluate the performance of these models individually and through a fusion approach, where their embeddings are combined for enhanced prediction.
  arXiv  Detail & Related papers  (2025-02-17T04:50:56Z)
• Electrocardiogram (ECG) Based Cardiac Arrhythmia Detection and Classification using Machine Learning Algorithms [0.0]
  Machine Learning (ML) and Deep Learning (DL) have opened new prospects in medical sciences for improved diagnosis, prognosis, and treatment of severe health conditions. This paper focuses on the development of an ML model with high predictive accuracy to classify arrhythmic electrocardiogram (ECG) signals.
  arXiv  Detail & Related papers  (2024-12-07T08:29:44Z)
• Self-supervised inter-intra period-aware ECG representation learning for detecting atrial fibrillation [41.82319894067087]
  We propose an inter-intra period-aware ECG representation learning approach. Considering ECGs of atrial fibrillation patients exhibit the irregularity in RR intervals and the absence of P-waves, we develop specific pre-training tasks for interperiod and intraperiod representations. Our approach demonstrates remarkable AUC performances on the BTCH dataset, textiti.e., 0.953/0.996 for paroxysmal/persistent atrial fibrillation detection.
  arXiv  Detail & Related papers  (2024-10-08T10:03:52Z)
• ECG Arrhythmia Detection Using Disease-specific Attention-based Deep Learning Model [0.0]
  We propose a disease-specific attention-based deep learning model (DANet) for arrhythmia detection from short ECG recordings. The novel idea is to introduce a soft-coding or hard-coding waveform enhanced module into existing deep neural networks. For the soft-coding DANet, we also develop a learning framework combining self-supervised pre-training with two-stage supervised training.
  arXiv  Detail & Related papers  (2024-07-25T13:27:10Z)
• TACCO: Task-guided Co-clustering of Clinical Concepts and Patient Visits for Disease Subtyping based on EHR Data [42.96821770394798]
  TACCO is a novel framework that jointly discovers clusters of clinical concepts and patient visits based on a hypergraph modeling of EHR data. We conduct experiments on the public MIMIC-III dataset and Emory internal CRADLE dataset over the downstream clinical tasks of phenotype classification and cardiovascular risk prediction. In-depth model analysis, clustering results analysis, and clinical case studies further validate the improved utilities and insightful interpretations delivered by TACCO.
  arXiv  Detail & Related papers  (2024-06-14T14:18:38Z)
• Self-Trained Model for ECG Complex Delineation [0.0]
  Electrocardiogram (ECG) delineation plays a crucial role in assisting cardiologists with accurate diagnoses. We introduce a dataset for ECG delineation and propose a novel self-trained method aimed at leveraging a vast amount of unlabeled ECG data. Our approach involves the pseudolabeling of unlabeled data using a neural network trained on our dataset. Subsequently, we train the model on the newly labeled samples to enhance the quality of delineation.
  arXiv  Detail & Related papers  (2024-06-04T18:54:10Z)
• MEIT: Multi-Modal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation [41.324530807795256]
  Electrocardiogram (ECG) is the primary non-invasive diagnostic tool for monitoring cardiac conditions. Recent studies have concentrated on classifying cardiac conditions using ECG data but have overlooked ECG report generation. We propose the Multimodal ECG Instruction Tuning (MEIT) framework, the first attempt to tackle ECG report generation with LLMs and multimodal instructions.
  arXiv  Detail & Related papers  (2024-03-07T23:20:56Z)
• Polar-Net: A Clinical-Friendly Model for Alzheimer's Disease Detection in OCTA Images [53.235117594102675]
  Optical Coherence Tomography Angiography is a promising tool for detecting Alzheimer's disease (AD) by imaging the retinal microvasculature. We propose a novel deep-learning framework called Polar-Net to provide interpretable results and leverage clinical prior knowledge. We show that Polar-Net outperforms existing state-of-the-art methods and provides more valuable pathological evidence for the association between retinal vascular changes and AD.
  arXiv  Detail & Related papers  (2023-11-10T11:49:49Z)
• Digital twinning of cardiac electrophysiology models from the surface ECG: a geodesic backpropagation approach [39.36827689390718]
  We introduce a novel method, Geodesic-BP, to solve the inverse eikonal problem. We show that Geodesic-BP can reconstruct a simulated cardiac activation with high accuracy in a synthetic test case. Given the future shift towards personalized medicine, Geodesic-BP has the potential to help in future functionalizations of cardiac models.
  arXiv  Detail & Related papers  (2023-08-16T14:57:12Z)
• Classification of lung cancer subtypes on CT images with synthetic pathological priors [41.75054301525535]
  Cross-scale associations exist in the image patterns between the same case's CT images and its pathological images. We propose self-generating hybrid feature network (SGHF-Net) for accurately classifying lung cancer subtypes on CT images.
  arXiv  Detail & Related papers  (2023-08-09T02:04:05Z)
• In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection [21.222167116069144]
  This paper presents a systematic investigation into the effectiveness of Self-Supervised Learning (SSL) methods for ECG arrhythmia detection. We begin by conducting a novel analysis of the data distributions on three popular ECG-based arrhythmia datasets: PTB-XL, Chapman, and Ribeiro. We then perform a comprehensive set of experiments using different augmentations and parameters to evaluate the effectiveness of various SSL methods. Our analysis shows that SSL methods achieve highly competitive results to those achieved by supervised state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-04-13T11:46:32Z)
• ECG-CL: A Comprehensive Electrocardiogram Interpretation Method Based on Continual Learning [20.465733855762835]
  Electrocardiogram (ECG) monitoring is one of the most powerful technique of cardiovascular disease (CVD) early identification. Classic rule-based algorithms are now completely outperformed by deep learning based methods. We propose a multi-resolution model that can sustain high-resolution low-level semantic information throughout.
  arXiv  Detail & Related papers  (2023-04-10T15:19:00Z)
• A Multi-View Learning Approach to Enhance Automatic 12-Lead ECG Diagnosis Performance [0.240137930000971]
  This study proposes an ensemble-based multi-view learning approach with an ECG augmentation technique to achieve a higher performance than traditional automatic 12-lead ECG diagnosis methods. The proposed model reports an F1 score of 0.840, which outperforms existing state-ofthe-art methods in the literature.
  arXiv  Detail & Related papers  (2022-07-30T23:43:01Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks [69.25956542388653]
  Deep learning (DL) algorithms are gaining weight in academic and industrial settings. We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework. The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
  arXiv  Detail & Related papers  (2020-05-11T16:29:12Z)
• Opportunities and Challenges of Deep Learning Methods for Electrocardiogram Data: A Systematic Review [62.490310870300746]
  The electrocardiogram (ECG) is one of the most commonly used diagnostic tools in medicine and healthcare. Deep learning methods have achieved promising results on predictive healthcare tasks using ECG signals. This paper presents a systematic review of deep learning methods for ECG data from both modeling and application perspectives.
  arXiv  Detail & Related papers  (2019-12-28T02:44:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.