Automated Cardiovascular Record Retrieval by Multimodal Learning between
Electrocardiogram and Clinical Report
- URL: http://arxiv.org/abs/2304.06286v3
- Date: Mon, 6 Nov 2023 18:31:34 GMT
- Title: Automated Cardiovascular Record Retrieval by Multimodal Learning between
Electrocardiogram and Clinical Report
- Authors: Jielin Qiu, Jiacheng Zhu, Shiqi Liu, William Han, Jingqi Zhang,
Chaojing Duan, Michael Rosenberg, Emerson Liu, Douglas Weber, Ding Zhao
- Abstract summary: We introduce a novel approach to ECG interpretation, leveraging recent breakthroughs in Large Language Models (LLMs) and Vision-Transformer (ViT) models.
We propose an alternative method of automatically identifying the most similar clinical cases based on the input ECG data.
Our findings could serve as a crucial resource for providing diagnostic services in underdeveloped regions.
- Score: 28.608260758775316
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Automated interpretation of electrocardiograms (ECG) has garnered significant
attention with the advancements in machine learning methodologies. Despite the
growing interest, most current studies focus solely on classification or
regression tasks, which overlook a crucial aspect of clinical cardio-disease
diagnosis: the diagnostic report generated by experienced human clinicians. In
this paper, we introduce a novel approach to ECG interpretation, leveraging
recent breakthroughs in Large Language Models (LLMs) and Vision-Transformer
(ViT) models. Rather than treating ECG diagnosis as a classification or
regression task, we propose an alternative method of automatically identifying
the most similar clinical cases based on the input ECG data. Also, since
interpreting ECG as images is more affordable and accessible, we process ECG as
encoded images and adopt a vision-language learning paradigm to jointly learn
vision-language alignment between encoded ECG images and ECG diagnosis reports.
Encoding ECG into images can result in an efficient ECG retrieval system, which
will be highly practical and useful in clinical applications. More importantly,
our findings could serve as a crucial resource for providing diagnostic
services in underdeveloped regions.
Related papers
- ECG Arrhythmia Detection Using Disease-specific Attention-based Deep Learning Model [0.0]
We propose a disease-specific attention-based deep learning model (DANet) for arrhythmia detection from short ECG recordings.
The novel idea is to introduce a soft-coding or hard-coding waveform enhanced module into existing deep neural networks.
For the soft-coding DANet, we also develop a learning framework combining self-supervised pre-training with two-stage supervised training.
arXiv Detail & Related papers (2024-07-25T13:27:10Z) - ECG Semantic Integrator (ESI): A Foundation ECG Model Pretrained with LLM-Enhanced Cardiological Text [14.06147507373525]
This study introduces a new multimodal contrastive pretaining framework that aims to improve the quality and robustness of learned representations of 12-lead ECG signals.
Our framework comprises two key components, including Cardio Query Assistant (CQA) and ECG Semantics Integrator(ESI)
arXiv Detail & Related papers (2024-05-26T06:45:39Z) - Polar-Net: A Clinical-Friendly Model for Alzheimer's Disease Detection
in OCTA Images [53.235117594102675]
Optical Coherence Tomography Angiography is a promising tool for detecting Alzheimer's disease (AD) by imaging the retinal microvasculature.
We propose a novel deep-learning framework called Polar-Net to provide interpretable results and leverage clinical prior knowledge.
We show that Polar-Net outperforms existing state-of-the-art methods and provides more valuable pathological evidence for the association between retinal vascular changes and AD.
arXiv Detail & Related papers (2023-11-10T11:49:49Z) - ECG-SL: Electrocardiogram(ECG) Segment Learning, a deep learning method
for ECG signal [19.885905393439014]
We propose a novel ECG-Segment based Learning (ECG-SL) framework to explicitly model the periodic nature of ECG signals.
Based on the structural features, a temporal model is designed to learn the temporal information for various clinical tasks.
The proposed method outperforms the baseline model and shows competitive performances compared with task-specific methods in three clinical applications.
arXiv Detail & Related papers (2023-10-01T23:17:55Z) - Unlocking the Diagnostic Potential of ECG through Knowledge Transfer
from Cardiac MRI [6.257859765229826]
We propose the first self-supervised contrastive approach that transfers domain-specific information from CMR images to ECG embeddings.
Our approach combines multimodal contrastive learning with masked data modeling to enable holistic cardiac screening solely from ECG data.
arXiv Detail & Related papers (2023-08-09T10:05:11Z) - Identifying Electrocardiogram Abnormalities Using a
Handcrafted-Rule-Enhanced Neural Network [18.859487271034336]
We introduce some rules into convolutional neural networks, which help present clinical knowledge to deep learning based ECG analysis.
Our new approach considerably outperforms existing state-of-the-art methods.
arXiv Detail & Related papers (2022-06-16T04:42:57Z) - Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation [116.87918100031153]
We propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG)
CGT injects clinical relation triples into the visual features as prior knowledge to drive the decoding procedure.
Experiments on the large-scale FFA-IR benchmark demonstrate that the proposed CGT is able to outperform previous benchmark methods.
arXiv Detail & Related papers (2022-06-04T13:16:30Z) - Semantic segmentation of multispectral photoacoustic images using deep
learning [53.65837038435433]
Photoacoustic imaging has the potential to revolutionise healthcare.
Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information.
We present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images.
arXiv Detail & Related papers (2021-05-20T09:33:55Z) - Explaining Clinical Decision Support Systems in Medical Imaging using
Cycle-Consistent Activation Maximization [112.2628296775395]
Clinical decision support using deep neural networks has become a topic of steadily growing interest.
clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend.
We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
arXiv Detail & Related papers (2020-10-09T14:39:27Z) - ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed
Quality Labeling Using Neural Networks [69.25956542388653]
Deep learning (DL) algorithms are gaining weight in academic and industrial settings.
We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework.
The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
arXiv Detail & Related papers (2020-05-11T16:29:12Z) - A Global Benchmark of Algorithms for Segmenting Late Gadolinium-Enhanced
Cardiac Magnetic Resonance Imaging [90.29017019187282]
" 2018 Left Atrium Challenge" using 154 3D LGE-MRIs, currently the world's largest cardiac LGE-MRI dataset.
Analyse of the submitted algorithms using technical and biological metrics was performed.
Results show the top method achieved a dice score of 93.2% and a mean surface to a surface distance of 0.7 mm.
arXiv Detail & Related papers (2020-04-26T08:49:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.