Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging and Clinical Data

• URL: http://arxiv.org/abs/2505.02677v1
• Date: Mon, 05 May 2025 14:22:58 GMT
• Title: Multimodal Deep Learning for Stroke Prediction and Detection using Retinal Imaging and Clinical Data
• Authors: Saeed Shurrab, Aadim Nepal, Terrence J. Lee-St. John, Nicola G. Ghazi, Bartlomiej Piechowski-Jozwiak, Farah E. Shamout,
• Abstract summary: This study explores the impact of leveraging retinal images and clinical data for stroke detection and risk prediction.<n>We propose a multimodal deep neural network that processes Optical Coherence Tomography ( OCT) and infrared reflectance retinal scans.<n>Our empirical findings establish the predictive ability of the considered modalities in detecting lasting effects in the retina associated with acute stroke.
• Score: 0.7322887425853788
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Stroke is a major public health problem, affecting millions worldwide. Deep learning has recently demonstrated promise for enhancing the diagnosis and risk prediction of stroke. However, existing methods rely on costly medical imaging modalities, such as computed tomography. Recent studies suggest that retinal imaging could offer a cost-effective alternative for cerebrovascular health assessment due to the shared clinical pathways between the retina and the brain. Hence, this study explores the impact of leveraging retinal images and clinical data for stroke detection and risk prediction. We propose a multimodal deep neural network that processes Optical Coherence Tomography (OCT) and infrared reflectance retinal scans, combined with clinical data, such as demographics, vital signs, and diagnosis codes. We pretrained our model using a self-supervised learning framework using a real-world dataset consisting of $37$ k scans, and then fine-tuned and evaluated the model using a smaller labeled subset. Our empirical findings establish the predictive ability of the considered modalities in detecting lasting effects in the retina associated with acute stroke and forecasting future risk within a specific time horizon. The experimental results demonstrate the effectiveness of our proposed framework by achieving $5$\% AUROC improvement as compared to the unimodal image-only baseline, and $8$\% improvement compared to an existing state-of-the-art foundation model. In conclusion, our study highlights the potential of retinal imaging in identifying high-risk patients and improving long-term outcomes.

Related papers

• Bayesian Deep Learning Approaches for Uncertainty-Aware Retinal OCT Image Segmentation for Multiple Sclerosis [0.0]
  Optical Coherence Tomography ( OCT) provides valuable insights in ophthalmology, cardiology, and neurology.<n>One critical task for ophthalmologists using OCT is delineation of retinal layers within scans.<n>Previous efforts to automate delineation using deep learning face challenges in uptake from clinicians and statisticians.
  arXiv  Detail & Related papers  (2025-05-17T15:56:17Z)
• Multi-task Learning Approach for Intracranial Hemorrhage Prognosis [0.0]
  We propose a 3D multi-task image model to predict prognosis, Glasgow Coma Scale and age, improving accuracy and interpretability. Our method outperforms current state-of-the-art baseline image models, and demonstrates superior performance in ICH prognosis compared to four board-certified neuroradiologists using only CT scans as input.
  arXiv  Detail & Related papers  (2024-08-16T14:56:17Z)
• Enhance Eye Disease Detection using Learnable Probabilistic Discrete Latents in Machine Learning Architectures [1.6000489723889526]
  Ocular diseases, including diabetic retinopathy and glaucoma, present a significant public health challenge.<n>Deep learning models have emerged as powerful tools for analysing medical images, such as retina imaging.<n>Challenges persist in model relibability and uncertainty estimation, which are critical for clinical decision-making.
  arXiv  Detail & Related papers  (2024-01-21T04:14:54Z)
• Detection and Classification of Diabetic Retinopathy using Deep Learning Algorithms for Segmentation to Facilitate Referral Recommendation for Test and Treatment Prediction [0.0]
  This research paper addresses the critical challenge of diabetic retinopathy (DR), a severe complication of diabetes leading to potential blindness. The proposed methodology leverages transfer learning with convolutional neural networks (CNNs) for automatic DR detection using a single fundus photograph. High evaluation scores in Jaccard, F1, recall, precision, and accuracy underscore the model's potential for enhancing diagnostic capabilities in retinal pathology assessment.
  arXiv  Detail & Related papers  (2024-01-05T11:19:24Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Computational Image-based Stroke Assessment for Evaluation of Cerebroprotectants with Longitudinal and Multi-site Preclinical MRI [0.4460373311150658]
  We developed, evaluated, and deployed a pipeline for image-based stroke outcome quantification for the Stroke Preclinical Assessment Network (SPAN) Our fully automated pipeline combines state-of-the-art algorithmic and data analytic approaches to assess stroke outcomes. Our results demonstrate the efficacy and robustness of our image-based stroke assessments.
  arXiv  Detail & Related papers  (2022-03-11T01:53:30Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images [72.94446225783697]
  We propose a machine learning system for the detection of referable Diabetic Retinopathy in fundus images. By extracting local information from image patches and combining it efficiently through an attention mechanism, our system is able to achieve high classification accuracy. We evaluate our approach on publicly available retinal image datasets, in which it exhibits near state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-03-02T13:14:15Z)
• Explaining Clinical Decision Support Systems in Medical Imaging using Cycle-Consistent Activation Maximization [112.2628296775395]
  Clinical decision support using deep neural networks has become a topic of steadily growing interest. clinicians are often hesitant to adopt the technology because its underlying decision-making process is considered to be intransparent and difficult to comprehend. We propose a novel decision explanation scheme based on CycleGAN activation which generates high-quality visualizations of classifier decisions even in smaller data sets.
  arXiv  Detail & Related papers  (2020-10-09T14:39:27Z)
• Modeling and Enhancing Low-quality Retinal Fundus Images [167.02325845822276]
  Low-quality fundus images increase uncertainty in clinical observation and lead to the risk of misdiagnosis. We propose a clinically oriented fundus enhancement network (cofe-Net) to suppress global degradation factors. Experiments on both synthetic and real images demonstrate that our algorithm effectively corrects low-quality fundus images without losing retinal details.
  arXiv  Detail & Related papers  (2020-05-12T08:01:16Z)
• Retinopathy of Prematurity Stage Diagnosis Using Object Segmentation and Convolutional Neural Networks [68.96150598294072]
  Retinopathy of Prematurity (ROP) is an eye disorder primarily affecting premature infants with lower weights. It causes proliferation of vessels in the retina and could result in vision loss and, eventually, retinal detachment, leading to blindness. In recent years, there has been a significant effort to automate the diagnosis using deep learning. This paper builds upon the success of previous models and develops a novel architecture, which combines object segmentation and convolutional neural networks (CNN) Our proposed system first trains an object segmentation model to identify the demarcation line at a pixel level and adds the resulting mask as an additional "color" channel in
  arXiv  Detail & Related papers  (2020-04-03T14:07:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.