CNN-LSTM Based Multimodal MRI and Clinical Data Fusion for Predicting Functional Outcome in Stroke Patients

• URL: http://arxiv.org/abs/2205.05545v1
• Date: Wed, 11 May 2022 14:46:01 GMT
• Title: CNN-LSTM Based Multimodal MRI and Clinical Data Fusion for Predicting Functional Outcome in Stroke Patients
• Authors: Nima Hatami and Tae-Hee Cho and Laura Mechtouff and Omer Faruk Eker and David Rousseau and Carole Frindel
• Abstract summary: Clinical outcome prediction plays an important role in stroke patient management. From a machine learning point-of-view, one of the main challenges is dealing with heterogeneous data. In this paper a multimodal convolutional neural network - long short-term memory (CNN-LSTM) based ensemble model is proposed.
• Score: 1.5250925845050138
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Clinical outcome prediction plays an important role in stroke patient management. From a machine learning point-of-view, one of the main challenges is dealing with heterogeneous data at patient admission, i.e. the image data which are multidimensional and the clinical data which are scalars. In this paper, a multimodal convolutional neural network - long short-term memory (CNN-LSTM) based ensemble model is proposed. For each MR image module, a dedicated network provides preliminary prediction of the clinical outcome using the modified Rankin scale (mRS). The final mRS score is obtained by merging the preliminary probabilities of each module dedicated to a specific type of MR image weighted by the clinical metadata, here age or the National Institutes of Health Stroke Scale (NIHSS). The experimental results demonstrate that the proposed model surpasses the baselines and offers an original way to automatically encode the spatio-temporal context of MR images in a deep learning architecture. The highest AUC (0.77) was achieved for the proposed model with NIHSS.

Related papers

• NeuroPictor: Refining fMRI-to-Image Reconstruction via Multi-individual Pretraining and Multi-level Modulation [55.51412454263856]
  This paper proposes to directly modulate the generation process of diffusion models using fMRI signals. By training with about 67,000 fMRI-image pairs from various individuals, our model enjoys superior fMRI-to-image decoding capacity.
  arXiv  Detail & Related papers  (2024-03-27T02:42:52Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation [5.662694302758443]
  Multi-sequence magnetic resonance imaging (MRI) has found wide applications in both modern clinical studies and deep learning research. It frequently occurs that one or more of the MRI sequences are missing due to different image acquisition protocols or contrast agent contraindications of patients. One promising approach is to leverage generative models to synthesize the missing sequences, which can serve as a surrogate acquisition.
  arXiv  Detail & Related papers  (2023-09-06T19:01:58Z)
• LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching [59.01894976615714]
  We introduce LVM-Med, the first family of deep networks trained on large-scale medical datasets. We have collected approximately 1.3 million medical images from 55 publicly available datasets. LVM-Med empirically outperforms a number of state-of-the-art supervised, self-supervised, and foundation models.
  arXiv  Detail & Related papers  (2023-06-20T22:21:34Z)
• A Light-weight CNN Model for Efficient Parkinson's Disease Diagnostics [1.382077805849933]
  The proposed model consists of a convolution neural network (CNN) to short-term memory (LSTM) to adapt the characteristics of collected time-series signals. Experimental results show that the proposed model achieves a high-quality diagnostic result over multiple evaluation metrics with much fewer parameters and operations.
  arXiv  Detail & Related papers  (2023-02-02T09:49:07Z)
• A multi-stream convolutional neural network for classification of progressive MCI in Alzheimer's disease using structural MRI images [0.23633885460047763]
  We propose a multi-stream deep convolutional neural network fed with patch-based imaging data to classify stable MCI and progressive MCI. First, we compare MRI images of Alzheimer's disease with cognitively normal subjects to identify distinct anatomical landmarks. These landmarks are then used to extract patches that are fed into the proposed multi-stream convolutional neural network to classify MRI images.
  arXiv  Detail & Related papers  (2022-03-03T15:14:13Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)
• Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning [62.17532253489087]
  Deep learning methods have been shown to produce superior performance on MR image reconstruction. These methods require large amounts of data which is difficult to collect and share due to the high cost of acquisition and medical data privacy regulations. We propose a federated learning (FL) based solution in which we take advantage of the MR data available at different institutions while preserving patients' privacy.
  arXiv  Detail & Related papers  (2021-03-03T03:04:40Z)
• Enhancing Fiber Orientation Distributions using convolutional Neural Networks [0.0]
  We learn improved FODs for commercially acquired MRI. We evaluate patch-based 3D convolutional neural networks (CNNs) Our approach may enable robust CSD model estimation on single-shell dMRI acquisition protocols.
  arXiv  Detail & Related papers  (2020-08-12T16:06:25Z)
• Prediction of Thrombectomy Functional Outcomes using Multimodal Data [2.358784542343728]
  We propose a novel deep learning approach to directly exploit multimodal data to estimate the success of endovascular treatment. We incorporate an attention mechanism in our architecture to model global feature inter-dependencies, both channel-wise and spatially.
  arXiv  Detail & Related papers  (2020-05-26T21:51:58Z)
• MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data [75.73881040581767]
  We propose a novel multi-site network (MS-Net) for improving prostate segmentation by learning robust representations. Our MS-Net improves the performance across all datasets consistently, and outperforms state-of-the-art methods for multi-site learning.
  arXiv  Detail & Related papers  (2020-02-09T14:11:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.