Frozen Language Model Helps ECG Zero-Shot Learning
- URL: http://arxiv.org/abs/2303.12311v1
- Date: Wed, 22 Mar 2023 05:01:14 GMT
- Title: Frozen Language Model Helps ECG Zero-Shot Learning
- Authors: Jun Li, Che Liu, Sibo Cheng, Rossella Arcucci, Shenda Hong
- Abstract summary: We propose Multimodal ECG-Text Self-supervised pre-training (METS)
We use a trainable ECG encoder and a frozen language model to embed paired ECG and automatically machine-generated clinical reports separately.
In downstream classification tasks, METS achieves around 10% improvement in performance without using any annotated data.
- Score: 12.974685769614062
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The electrocardiogram (ECG) is one of the most commonly used non-invasive,
convenient medical monitoring tools that assist in the clinical diagnosis of
heart diseases. Recently, deep learning (DL) techniques, particularly
self-supervised learning (SSL), have demonstrated great potential in the
classification of ECG. SSL pre-training has achieved competitive performance
with only a small amount of annotated data after fine-tuning. However, current
SSL methods rely on the availability of annotated data and are unable to
predict labels not existing in fine-tuning datasets. To address this challenge,
we propose Multimodal ECG-Text Self-supervised pre-training (METS), the first
work to utilize the auto-generated clinical reports to guide ECG SSL
pre-training. We use a trainable ECG encoder and a frozen language model to
embed paired ECG and automatically machine-generated clinical reports
separately. The SSL aims to maximize the similarity between paired ECG and
auto-generated report while minimize the similarity between ECG and other
reports. In downstream classification tasks, METS achieves around 10%
improvement in performance without using any annotated data via zero-shot
classification, compared to other supervised and SSL baselines that rely on
annotated data. Furthermore, METS achieves the highest recall and F1 scores on
the MIT-BIH dataset, despite MIT-BIH containing different classes of ECG
compared to the pre-trained dataset. The extensive experiments have
demonstrated the advantages of using ECG-Text multimodal self-supervised
learning in terms of generalizability, effectiveness, and efficiency.
Related papers
- Zero-Shot ECG Classification with Multimodal Learning and Test-time Clinical Knowledge Enhancement [10.611952462532908]
Multimodal ECG Representation Learning (MERL) is capable of performing zero-shot ECG classification with text prompts.
We propose the Clinical Knowledge Enhanced Prompt Engineering (CKEPE) approach to exploit external expert-verified clinical knowledge databases.
MERL achieves an average AUC score of 75.2% in zero-shot classification (without training data), 3.2% higher than linear probed eSSL methods with 10% annotated training data, averaged across all six datasets.
arXiv Detail & Related papers (2024-03-11T12:28:55Z) - MEIT: Multi-Modal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation [41.324530807795256]
Electrocardiogram (ECG) is the primary non-invasive diagnostic tool for monitoring cardiac conditions.
Recent studies have concentrated on classifying cardiac conditions using ECG data but have overlooked ECG report generation.
We propose the Multimodal ECG Instruction Tuning (MEIT) framework, the first attempt to tackle ECG report generation with LLMs and multimodal instructions.
arXiv Detail & Related papers (2024-03-07T23:20:56Z) - Guiding Masked Representation Learning to Capture Spatio-Temporal Relationship of Electrocardiogram [2.2842904085777045]
We introduce ST-MEM (S-Temporal Masked Electrocardiogram Modeling), designed to learntemporal features by reconstructing 12-lead ECG data.
ST-MEM outperforms other SSL baseline methods in various experimental settings for arrhythmia.
arXiv Detail & Related papers (2024-02-02T10:04:13Z) - ECG-SL: Electrocardiogram(ECG) Segment Learning, a deep learning method
for ECG signal [19.885905393439014]
We propose a novel ECG-Segment based Learning (ECG-SL) framework to explicitly model the periodic nature of ECG signals.
Based on the structural features, a temporal model is designed to learn the temporal information for various clinical tasks.
The proposed method outperforms the baseline model and shows competitive performances compared with task-specific methods in three clinical applications.
arXiv Detail & Related papers (2023-10-01T23:17:55Z) - ETP: Learning Transferable ECG Representations via ECG-Text Pre-training [10.856365645831728]
ECG-Text Pre-training (ETP) is an innovative framework designed to learn cross-modal representations that link ECG signals with textual reports.
ETP employs an ECG encoder along with a pre-trained language model to align ECG signals with their corresponding textual reports.
arXiv Detail & Related papers (2023-09-06T19:19:26Z) - Improving Multiple Sclerosis Lesion Segmentation Across Clinical Sites:
A Federated Learning Approach with Noise-Resilient Training [75.40980802817349]
Deep learning models have shown promise for automatically segmenting MS lesions, but the scarcity of accurately annotated data hinders progress in this area.
We introduce a Decoupled Hard Label Correction (DHLC) strategy that considers the imbalanced distribution and fuzzy boundaries of MS lesions.
We also introduce a Centrally Enhanced Label Correction (CELC) strategy, which leverages the aggregated central model as a correction teacher for all sites.
arXiv Detail & Related papers (2023-08-31T00:36:10Z) - Learnable Weight Initialization for Volumetric Medical Image Segmentation [66.3030435676252]
We propose a learnable weight-based hybrid medical image segmentation approach.
Our approach is easy to integrate into any hybrid model and requires no external training data.
Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach.
arXiv Detail & Related papers (2023-06-15T17:55:05Z) - Self-supervised contrastive learning of echocardiogram videos enables
label-efficient cardiac disease diagnosis [48.64462717254158]
We developed a self-supervised contrastive learning approach, EchoCLR, to catered to echocardiogram videos.
When fine-tuned on small portions of labeled data, EchoCLR pretraining significantly improved classification performance for left ventricular hypertrophy (LVH) and aortic stenosis (AS)
EchoCLR is unique in its ability to learn representations of medical videos and demonstrates that SSL can enable label-efficient disease classification from small, labeled datasets.
arXiv Detail & Related papers (2022-07-23T19:17:26Z) - Uncovering the structure of clinical EEG signals with self-supervised
learning [64.4754948595556]
Supervised learning paradigms are often limited by the amount of labeled data that is available.
This phenomenon is particularly problematic in clinically-relevant data, such as electroencephalography (EEG)
By extracting information from unlabeled data, it might be possible to reach competitive performance with deep neural networks.
arXiv Detail & Related papers (2020-07-31T14:34:47Z) - ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed
Quality Labeling Using Neural Networks [69.25956542388653]
Deep learning (DL) algorithms are gaining weight in academic and industrial settings.
We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework.
The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
arXiv Detail & Related papers (2020-05-11T16:29:12Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.