Enhancing Cardiovascular Disease Prediction through Multi-Modal Self-Supervised Learning

• URL: http://arxiv.org/abs/2411.05900v1
• Date: Fri, 08 Nov 2024 16:32:30 GMT
• Title: Enhancing Cardiovascular Disease Prediction through Multi-Modal Self-Supervised Learning
• Authors: Francesco Girlanda, Olga Demler, Bjoern Menze, Neda Davoudi,
• Abstract summary: We propose a comprehensive framework for enhancing cardiovascular disease prediction with limited annotated datasets. We employ a masked autoencoder to pre-train the electrocardiogram ECG encoder, enabling it to extract relevant features from raw electrocardiogram data. We fine-tuned the pre-trained encoders on specific predictive tasks, such as myocardial infarction.
• Score: 0.17708284654788597
• License:
• Abstract: Accurate prediction of cardiovascular diseases remains imperative for early diagnosis and intervention, necessitating robust and precise predictive models. Recently, there has been a growing interest in multi-modal learning for uncovering novel insights not available through uni-modal datasets alone. By combining cardiac magnetic resonance images, electrocardiogram signals, and available medical information, our approach enables the capture of holistic status about individuals' cardiovascular health by leveraging shared information across modalities. Integrating information from multiple modalities and benefiting from self-supervised learning techniques, our model provides a comprehensive framework for enhancing cardiovascular disease prediction with limited annotated datasets. We employ a masked autoencoder to pre-train the electrocardiogram ECG encoder, enabling it to extract relevant features from raw electrocardiogram data, and an image encoder to extract relevant features from cardiac magnetic resonance images. Subsequently, we utilize a multi-modal contrastive learning objective to transfer knowledge from expensive and complex modality, cardiac magnetic resonance image, to cheap and simple modalities such as electrocardiograms and medical information. Finally, we fine-tuned the pre-trained encoders on specific predictive tasks, such as myocardial infarction. Our proposed method enhanced the image information by leveraging different available modalities and outperformed the supervised approach by 7.6% in balanced accuracy.

Related papers

• CognitionCapturer: Decoding Visual Stimuli From Human EEG Signal With Multimodal Information [61.1904164368732]
  We propose CognitionCapturer, a unified framework that fully leverages multimodal data to represent EEG signals. Specifically, CognitionCapturer trains Modality Experts for each modality to extract cross-modal information from the EEG modality. The framework does not require any fine-tuning of the generative models and can be extended to incorporate more modalities.
  arXiv  Detail & Related papers  (2024-12-13T16:27:54Z)
• Private, Efficient and Scalable Kernel Learning for Medical Image Analysis [1.7999333451993955]
  OKRA (Orthonormal K-fRAmes) is a novel randomized encoding-based approach for kernel-based machine learning. It significantly enhances scalability and speed compared to current state-of-the-art solutions.
  arXiv  Detail & Related papers  (2024-10-21T10:03:03Z)
• CMRxRecon2024: A Multi-Modality, Multi-View K-Space Dataset Boosting Universal Machine Learning for Accelerated Cardiac MRI [40.11088079783521]
  The CMRxRecon2024 dataset is the largest and most protocal-diverse publicly available cardiac k-space dataset. It is acquired from 330 healthy volunteers, covering commonly used modalities, anatomical views, and acquisition trajectories in clinical cardiac MRI.
  arXiv  Detail & Related papers  (2024-06-27T09:50:20Z)
• Deciphering Heartbeat Signatures: A Vision Transformer Approach to Explainable Atrial Fibrillation Detection from ECG Signals [4.056982620027252]
  We develop a vision transformer approach to identify atrial fibrillation based on single-lead ECG data. A residual network (ResNet) approach is also developed for comparison with the vision transformer approach.
  arXiv  Detail & Related papers  (2024-02-12T11:04:08Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• Digital twinning of cardiac electrophysiology models from the surface ECG: a geodesic backpropagation approach [39.36827689390718]
  We introduce a novel method, Geodesic-BP, to solve the inverse eikonal problem. We show that Geodesic-BP can reconstruct a simulated cardiac activation with high accuracy in a synthetic test case. Given the future shift towards personalized medicine, Geodesic-BP has the potential to help in future functionalizations of cardiac models.
  arXiv  Detail & Related papers  (2023-08-16T14:57:12Z)
• Unlocking the diagnostic potential of electrocardiograms through information transfer from cardiac magnetic resonance imaging [9.77594028637426]
  We present a deep learning strategy for cost-effective and comprehensive cardiac screening solely from ECG. Our approach combines multimodal contrastive learning with masked data modelling to transfer domain-specific information from CMR imaging to ECG representations.
  arXiv  Detail & Related papers  (2023-08-09T10:05:11Z)
• ECG-Based Heart Arrhythmia Diagnosis Through Attentional Convolutional Neural Networks [9.410102957429705]
  We propose Attention-Based Convolutional Neural Networks (ABCNN) to work on the raw ECG signals and automatically extract the informative dependencies for accurate arrhythmia detection. Our main task is to find the arrhythmia from normal heartbeats and, at the meantime, accurately recognize the heart diseases from five arrhythmia types. The experimental results show that the proposed ABCNN outperforms the widely used baselines.
  arXiv  Detail & Related papers  (2021-08-18T14:55:46Z)
• Cross-Modal Information Maximization for Medical Imaging: CMIM [62.28852442561818]
  In hospitals, data are siloed to specific information systems that make the same information available under different modalities. This offers unique opportunities to obtain and use at train-time those multiple views of the same information that might not always be available at test-time. We propose an innovative framework that makes the most of available data by learning good representations of a multi-modal input that are resilient to modality dropping at test-time.
  arXiv  Detail & Related papers  (2020-10-20T20:05:35Z)
• Heart Sound Segmentation using Bidirectional LSTMs with Attention [37.62160903348547]
  We propose a novel framework for the segmentation of phonocardiogram (PCG) signals into heart states. We exploit recent advancements in attention based learning to segment the PCG signal. The proposed method attains state-of-the-art performance on multiple benchmarks including both human and animal heart recordings.
  arXiv  Detail & Related papers  (2020-04-02T02:09:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.