Enhancing Cardiovascular Disease Prediction through Multi-Modal Self-Supervised Learning

• URL: http://arxiv.org/abs/2411.05900v1
• Date: Fri, 08 Nov 2024 16:32:30 GMT
• Title: Enhancing Cardiovascular Disease Prediction through Multi-Modal Self-Supervised Learning
• Authors: Francesco Girlanda, Olga Demler, Bjoern Menze, Neda Davoudi,
• Abstract summary: We propose a comprehensive framework for enhancing cardiovascular disease prediction with limited annotated datasets. We employ a masked autoencoder to pre-train the electrocardiogram ECG encoder, enabling it to extract relevant features from raw electrocardiogram data. We fine-tuned the pre-trained encoders on specific predictive tasks, such as myocardial infarction.
• Score: 0.17708284654788597
• License:
• Abstract: Accurate prediction of cardiovascular diseases remains imperative for early diagnosis and intervention, necessitating robust and precise predictive models. Recently, there has been a growing interest in multi-modal learning for uncovering novel insights not available through uni-modal datasets alone. By combining cardiac magnetic resonance images, electrocardiogram signals, and available medical information, our approach enables the capture of holistic status about individuals' cardiovascular health by leveraging shared information across modalities. Integrating information from multiple modalities and benefiting from self-supervised learning techniques, our model provides a comprehensive framework for enhancing cardiovascular disease prediction with limited annotated datasets. We employ a masked autoencoder to pre-train the electrocardiogram ECG encoder, enabling it to extract relevant features from raw electrocardiogram data, and an image encoder to extract relevant features from cardiac magnetic resonance images. Subsequently, we utilize a multi-modal contrastive learning objective to transfer knowledge from expensive and complex modality, cardiac magnetic resonance image, to cheap and simple modalities such as electrocardiograms and medical information. Finally, we fine-tuned the pre-trained encoders on specific predictive tasks, such as myocardial infarction. Our proposed method enhanced the image information by leveraging different available modalities and outperformed the supervised approach by 7.6% in balanced accuracy.

Related papers

• Private, Efficient and Scalable Kernel Learning for Medical Image Analysis [1.7999333451993955]
  OKRA (Orthonormal K-fRAmes) is a novel randomized encoding-based approach for kernel-based machine learning. It significantly enhances scalability and speed compared to current state-of-the-art solutions.
  arXiv  Detail & Related papers  (2024-10-21T10:03:03Z)
• CMRxRecon2024: A Multi-Modality, Multi-View K-Space Dataset Boosting Universal Machine Learning for Accelerated Cardiac MRI [39.0162369912624]
  The CMRxRecon2024 dataset is the largest and most diverse publicly available cardiac k-space dataset. It is acquired from 330 healthy volunteers, covering commonly used modalities, anatomical views, and acquisition trajectories in clinical cardiac MRI.
  arXiv  Detail & Related papers  (2024-06-27T09:50:20Z)
• Deciphering Heartbeat Signatures: A Vision Transformer Approach to Explainable Atrial Fibrillation Detection from ECG Signals [4.056982620027252]
  We develop a vision transformer approach to identify atrial fibrillation based on single-lead ECG data. A residual network (ResNet) approach is also developed for comparison with the vision transformer approach.
  arXiv  Detail & Related papers  (2024-02-12T11:04:08Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Beyond Images: An Integrative Multi-modal Approach to Chest X-Ray Report Generation [47.250147322130545]
  Image-to-text radiology report generation aims to automatically produce radiology reports that describe the findings in medical images. Most existing methods focus solely on the image data, disregarding the other patient information accessible to radiologists. We present a novel multi-modal deep neural network framework for generating chest X-rays reports by integrating structured patient data, such as vital signs and symptoms, alongside unstructured clinical notes.
  arXiv  Detail & Related papers  (2023-11-18T14:37:53Z)
• Digital twinning of cardiac electrophysiology models from the surface ECG: a geodesic backpropagation approach [39.36827689390718]
  We introduce a novel method, Geodesic-BP, to solve the inverse eikonal problem. We show that Geodesic-BP can reconstruct a simulated cardiac activation with high accuracy in a synthetic test case. Given the future shift towards personalized medicine, Geodesic-BP has the potential to help in future functionalizations of cardiac models.
  arXiv  Detail & Related papers  (2023-08-16T14:57:12Z)
• Unlocking the Diagnostic Potential of ECG through Knowledge Transfer from Cardiac MRI [6.257859765229826]
  We propose the first self-supervised contrastive approach that transfers domain-specific information from CMR images to ECG embeddings. Our approach combines multimodal contrastive learning with masked data modeling to enable holistic cardiac screening solely from ECG data.
  arXiv  Detail & Related papers  (2023-08-09T10:05:11Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks [9.410102957429705]
  We propose Attention-Based Convolutional Neural Networks (ABCNN) to work on the raw ECG signals and automatically extract the informative dependencies for accurate arrhythmia detection. Our main task is to find the arrhythmia from normal heartbeats and, at the meantime, accurately recognize the heart diseases from five arrhythmia types. The experimental results show that the proposed ABCNN outperforms the widely used baselines.
  arXiv  Detail & Related papers  (2021-08-18T14:55:46Z)
• Cross-Modal Information Maximization for Medical Imaging: CMIM [62.28852442561818]
  In hospitals, data are siloed to specific information systems that make the same information available under different modalities. This offers unique opportunities to obtain and use at train-time those multiple views of the same information that might not always be available at test-time. We propose an innovative framework that makes the most of available data by learning good representations of a multi-modal input that are resilient to modality dropping at test-time.
  arXiv  Detail & Related papers  (2020-10-20T20:05:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.