C-MELT: Contrastive Enhanced Masked Auto-Encoders for ECG-Language Pre-Training

• URL: http://arxiv.org/abs/2410.02131v2
• Date: Fri, 4 Oct 2024 11:05:18 GMT
• Title: C-MELT: Contrastive Enhanced Masked Auto-Encoders for ECG-Language Pre-Training
• Authors: Manh Pham, Aaqib Saeed, Dong Ma,
• Abstract summary: We propose C-MELT, a framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture. C-MELT uniquely combines the strengths of generative with enhanced discriminative capabilities to achieve robust cross-modal representations.
• Score: 10.088785685439134
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate interpretation of Electrocardiogram (ECG) signals is pivotal for diagnosing cardiovascular diseases. Integrating ECG signals with their accompanying textual reports holds immense potential to enhance clinical diagnostics through the combination of physiological data and qualitative insights. However, this integration faces significant challenges due to inherent modality disparities and the scarcity of labeled data for robust cross-modal learning. To address these obstacles, we propose C-MELT, a novel framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture. C-MELT uniquely combines the strengths of generative with enhanced discriminative capabilities to achieve robust cross-modal representations. This is accomplished through masked modality modeling, specialized loss functions, and an improved negative sampling strategy tailored for cross-modal alignment. Extensive experiments on five public datasets across diverse downstream tasks demonstrate that C-MELT significantly outperforms existing methods, achieving 15% and 2% increases in linear probing and zero-shot performance over state-of-the-art models, respectively. These results highlight the effectiveness of C-MELT, underscoring its potential to advance automated clinical diagnostics through multi-modal representations.

Related papers

• GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images [43.65650710265957]
  We introduce GEM, the first MLLM unifying ECG time series, 12-lead ECG images and text for grounded and clinician-aligned ECG interpretation. GEM enables feature-grounded analysis, evidence-driven reasoning, and a clinician-like diagnostic process through three core innovations. We propose the Grounded ECG task, a clinically motivated benchmark designed to assess the MLLM's capability in grounded ECG understanding.
  arXiv  Detail & Related papers  (2025-03-08T05:48:53Z)
• Conditional Electrocardiogram Generation Using Hierarchical Variational Autoencoders [0.0]
  We propose a conditional Nouveau VAE model for ECG signal generation (cNVAE-ECG) This paper proposes the publicly available conditional Nouveau VAE model for ECG signal generation (cNVAE-ECG)
  arXiv  Detail & Related papers  (2025-03-03T13:30:36Z)
• DiffuSETS: 12-lead ECG Generation Conditioned on Clinical Text Reports and Patient-Specific Information [13.680337221159506]
  Heart disease remains a significant threat to human health. Scarcity of high-quality ECG data, driven by privacy concerns and limited medical resources, creates a pressing need for effective ECG signal generation. We propose DiffuSETS, a novel framework capable of generating ECG signals with high semantic alignment and fidelity.
  arXiv  Detail & Related papers  (2025-01-10T12:55:34Z)
• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals. Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality. The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• ICH-SCNet: Intracerebral Hemorrhage Segmentation and Prognosis Classification Network Using CLIP-guided SAM mechanism [12.469269425813607]
  Intracerebral hemorrhage (ICH) is the most fatal subtype of stroke and is characterized by a high incidence of disability. Existing approaches address these two tasks independently and predominantly focus on imaging data alone. This paper introduces a multi-task network, ICH-SCNet, designed for both ICH segmentation and prognosis classification.
  arXiv  Detail & Related papers  (2024-11-07T12:34:25Z)
• Clinical Evaluation of Medical Image Synthesis: A Case Study in Wireless Capsule Endoscopy [63.39037092484374]
  Synthetic Data Generation based on Artificial Intelligence (AI) can transform the way clinical medicine is delivered. This study focuses on the clinical evaluation of medical SDG, with a proof-of-concept investigation on diagnosing Inflammatory Bowel Disease (IBD) using Wireless Capsule Endoscopy (WCE) images. The results show that TIDE-II generates clinically plausible, very realistic WCE images, of improved quality compared to relevant state-of-the-art generative models.
  arXiv  Detail & Related papers  (2024-10-31T19:48:50Z)
• PMT: Progressive Mean Teacher via Exploring Temporal Consistency for Semi-Supervised Medical Image Segmentation [51.509573838103854]
  We propose a semi-supervised learning framework, termed Progressive Mean Teachers (PMT), for medical image segmentation. Our PMT generates high-fidelity pseudo labels by learning robust and diverse features in the training process. Experimental results on two datasets with different modalities, i.e., CT and MRI, demonstrate that our method outperforms the state-of-the-art medical image segmentation approaches.
  arXiv  Detail & Related papers  (2024-09-08T15:02:25Z)
• TACCO: Task-guided Co-clustering of Clinical Concepts and Patient Visits for Disease Subtyping based on EHR Data [42.96821770394798]
  TACCO is a novel framework that jointly discovers clusters of clinical concepts and patient visits based on a hypergraph modeling of EHR data. We conduct experiments on the public MIMIC-III dataset and Emory internal CRADLE dataset over the downstream clinical tasks of phenotype classification and cardiovascular risk prediction. In-depth model analysis, clustering results analysis, and clinical case studies further validate the improved utilities and insightful interpretations delivered by TACCO.
  arXiv  Detail & Related papers  (2024-06-14T14:18:38Z)
• MEDBind: Unifying Language and Multimodal Medical Data Embeddings [18.954939735299963]
  We present MEDBind (Medical Electronic patient recorD), which learns joint embeddings across CXR, ECG, and medical text. Using text data as the central anchor, MEDBind features tri-modality binding, delivering competitive performance in top-K retrieval, zero-shot, and few-shot benchmarks.
  arXiv  Detail & Related papers  (2024-03-19T16:46:29Z)
• MM-SurvNet: Deep Learning-Based Survival Risk Stratification in Breast Cancer Through Multimodal Data Fusion [18.395418853966266]
  We propose a novel deep learning approach for breast cancer survival risk stratification. We employ vision transformers, specifically the MaxViT model, for image feature extraction, and self-attention to capture intricate image relationships at the patient level. A dual cross-attention mechanism fuses these features with genetic data, while clinical data is incorporated at the final layer to enhance predictive accuracy.
  arXiv  Detail & Related papers  (2024-02-19T02:31:36Z)
• Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites: A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
  Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area. We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions. We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
  arXiv  Detail & Related papers  (2023-08-31T00:36:10Z)
• Semi-Supervised Learning for Multi-Label Cardiovascular Diseases Prediction:A Multi-Dataset Study [17.84069222975825]
  Current ECG-based diagnosis systems show promising performance owing to the rapid development of deep learning techniques. Label scarcity problem, the co-occurrence of multiple CVDs and the poor performance on unseen datasets hinder the widespread application of deep learning-based models. We propose a multi-label semi-supervised model (ECGMatch) to recognize multiple CVDs simultaneously with limited supervision.
  arXiv  Detail & Related papers  (2023-06-18T07:46:19Z)
• A Dual-scale Lead-seperated Transformer With Lead-orthogonal Attention And Meta-information For Ecg Classification [26.07181634056045]
  This work proposes a dual-scale lead-separated transformer with lead-orthogonal attention and meta-information (DLTM-ECG) ECG segments are interpreted as independent patches, and together with the reduced dimension signal, they form a dual-scale representation. Our work has the potential for similar multichannel bioelectrical signal processing and physiological multimodal tasks.
  arXiv  Detail & Related papers  (2022-11-23T08:45:34Z)
• Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images [49.1861463923357]
  We propose an uncertainty-guided dual-consistency learning network (UDC-Net) for semi-supervised COVID-19 lesion segmentation from CT images. Our proposed UDC-Net improves the fully supervised method by 6.3% in Dice and outperforms other competitive semi-supervised approaches by significant margins.
  arXiv  Detail & Related papers  (2021-04-07T16:23:35Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)
• COVI-AgentSim: an Agent-based Model for Evaluating Methods of Digital Contact Tracing [68.68882022019272]
  COVI-AgentSim is an agent-based compartmental simulator based on virology, disease progression, social contact networks, and mobility patterns. We use COVI-AgentSim to perform cost-adjusted analyses comparing no DCT to: 1) standard binary contact tracing (BCT) that assigns binary recommendations based on binary test results; and 2) a rule-based method for feature-based contact tracing (FCT) that assigns a graded level of recommendation based on diverse individual features.
  arXiv  Detail & Related papers  (2020-10-30T00:47:01Z)
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks [69.25956542388653]
  Deep learning (DL) algorithms are gaining weight in academic and industrial settings. We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework. The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
  arXiv  Detail & Related papers  (2020-05-11T16:29:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.