Contrastive Cross-Modal Learning for Infusing Chest X-ray Knowledge into ECGs

• URL: http://arxiv.org/abs/2506.19329v1
• Date: Tue, 24 Jun 2025 05:47:26 GMT
• Title: Contrastive Cross-Modal Learning for Infusing Chest X-ray Knowledge into ECGs
• Authors: Vineet Punyamoorty, Aditya Malusare, Vaneet Aggarwal,
• Abstract summary: electrocardiograms (ECGs) and chest X-rays (CXRs) are two of the most widely used modalities for cardiac assessment.<n>In this work, we propose CroMoTEX, a novel contrastive learning-based framework that leverages chest X-rays during training to learn clinically informative ECG representations.
• Score: 29.705337940879705
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Modern diagnostic workflows are increasingly multimodal, integrating diverse data sources such as medical images, structured records, and physiological time series. Among these, electrocardiograms (ECGs) and chest X-rays (CXRs) are two of the most widely used modalities for cardiac assessment. While CXRs provide rich diagnostic information, ECGs are more accessible and can support scalable early warning systems. In this work, we propose CroMoTEX, a novel contrastive learning-based framework that leverages chest X-rays during training to learn clinically informative ECG representations for multiple cardiac-related pathologies: cardiomegaly, pleural effusion, and edema. Our method aligns ECG and CXR representations using a novel supervised cross-modal contrastive objective with adaptive hard negative weighting, enabling robust and task-relevant feature learning. At test time, CroMoTEX relies solely on ECG input, allowing scalable deployment in real-world settings where CXRs may be unavailable. Evaluated on the large-scale MIMIC-IV-ECG and MIMIC-CXR datasets, CroMoTEX outperforms baselines across all three pathologies, achieving up to 78.31 AUROC on edema. Our code is available at github.com/vineetpmoorty/cromotex.

Related papers

• Global and Local Contrastive Learning for Joint Representations from Cardiac MRI and ECG [40.407824759778784]
  PTACL (Patient and Temporal Alignment Contrastive Learning) is a multimodal contrastive learning framework that enhances ECG representations by integrating-temporal information from CMR.<n>We evaluate PTACL on paired ECG-CMR data from 27,951 subjects in the UK Biobank.<n>Our results highlight the potential of PTACL to enhance non-invasive cardiac diagnostics using ECG.
  arXiv  Detail & Related papers  (2025-06-24T17:19:39Z)
• From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining [22.214252217020174]
  We introduce MELP, a novel Multi-scale ECG-Language Pretraining (MELP) model that fully leverages hierarchical supervision from ECG-text pairs.<n>We evaluate MELP on three public ECG datasets across multiple tasks, including zero-shot ECG classification, linear probing, and transfer learning.
  arXiv  Detail & Related papers  (2025-06-11T07:22:17Z)
• Heartcare Suite: Multi-dimensional Understanding of ECG with Raw Multi-lead Signal Modeling [50.58126509704037]
  Heartcare Suite is a framework for fine-grained electrocardiogram (ECG) understanding.<n>Heartcare-220K is a high-quality, structured, and comprehensive multimodal ECG dataset.<n>Heartcare-Bench is a benchmark to guide the optimization of Medical Multimodal Large Language Models (Med-MLLMs) in ECG scenarios.
  arXiv  Detail & Related papers  (2025-06-06T07:56:41Z)
• xLSTM-ECG: Multi-label ECG Classification via Feature Fusion with xLSTM [14.02717596836022]
  We propose xLSTM-ECG, a novel approach for multi-label classification of ECG signals.<n>To the best of our knowledge, this work represents the first design and application of xLSTM modules specifically adapted for multi-label ECG classification.
  arXiv  Detail & Related papers  (2025-04-14T16:12:46Z)
• GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images [43.65650710265957]
  We introduce GEM, the first MLLM unifying ECG time series, 12-lead ECG images and text for grounded and clinician-aligned ECG interpretation.<n> GEM enables feature-grounded analysis, evidence-driven reasoning, and a clinician-like diagnostic process through three core innovations.<n>We propose the Grounded ECG task, a clinically motivated benchmark designed to assess the MLLM's capability in grounded ECG understanding.
  arXiv  Detail & Related papers  (2025-03-08T05:48:53Z)
• AnyECG: Foundational Models for Multitask Cardiac Analysis in Real-World Settings [34.078819572852446]
  Electrocardiogram (ECG) is highly sensitive in detecting acute heart attacks.<n>This paper introduces AnyECG, a foundational model designed to extract robust representations from any real-world ECG data.
  arXiv  Detail & Related papers  (2024-11-17T17:32:58Z)
• MEIT: Multi-Modal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation [41.324530807795256]
  Electrocardiogram (ECG) is the primary non-invasive diagnostic tool for monitoring cardiac conditions. Recent studies have concentrated on classifying cardiac conditions using ECG data but have overlooked ECG report generation. We propose the Multimodal ECG Instruction Tuning (MEIT) framework, the first attempt to tackle ECG report generation with LLMs and multimodal instructions.
  arXiv  Detail & Related papers  (2024-03-07T23:20:56Z)
• Unlocking the diagnostic potential of electrocardiograms through information transfer from cardiac magnetic resonance imaging [9.77594028637426]
  We present a deep learning strategy for cost-effective and comprehensive cardiac screening solely from ECG.<n>Our approach combines multimodal contrastive learning with masked data modelling to transfer domain-specific information from CMR imaging to ECG representations.
  arXiv  Detail & Related papers  (2023-08-09T10:05:11Z)
• Multi-scale Cross-restoration Framework for Electrocardiogram Anomaly Detection [33.48389041651675]
  Electrocardiogram (ECG) is a widely used diagnostic tool for detecting heart conditions. Rare cardiac diseases may be underdiagnosed using traditional ECG analysis, considering that no training dataset can exhaust all possible cardiac disorders. This paper proposes using anomaly detection to identify any unhealthy status, with normal ECGs solely for training.
  arXiv  Detail & Related papers  (2023-08-03T09:16:57Z)
• Optimising Chest X-Rays for Image Analysis by Identifying and Removing Confounding Factors [49.005337470305584]
  During the COVID-19 pandemic, the sheer volume of imaging performed in an emergency setting for COVID-19 diagnosis has resulted in a wide variability of clinical CXR acquisitions. The variable quality of clinically-acquired CXRs within publicly available datasets could have a profound effect on algorithm performance. We propose a simple and effective step-wise approach to pre-processing a COVID-19 chest X-ray dataset to remove undesired biases.
  arXiv  Detail & Related papers  (2022-08-22T13:57:04Z)
• A Global Benchmark of Algorithms for Segmenting Late Gadolinium-Enhanced Cardiac Magnetic Resonance Imaging [90.29017019187282]
  " 2018 Left Atrium Challenge" using 154 3D LGE-MRIs, currently the world's largest cardiac LGE-MRI dataset. Analyse of the submitted algorithms using technical and biological metrics was performed. Results show the top method achieved a dice score of 93.2% and a mean surface to a surface distance of 0.7 mm.
  arXiv  Detail & Related papers  (2020-04-26T08:49:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.