Towards dynamic multi-modal phenotyping using chest radiographs and physiological data

• URL: http://arxiv.org/abs/2111.02710v1
• Date: Thu, 4 Nov 2021 09:41:00 GMT
• Title: Towards dynamic multi-modal phenotyping using chest radiographs and physiological data
• Authors: Nasir Hayat, Krzysztof J. Geras, Farah E. Shamout
• Abstract summary: We propose a dynamic training approach to learn modality-specific data representations and to integrate auxiliary features. Preliminary experiments results for a patient phenotyping task using physiological data in MIMIC-IV & chest radiographs in the MIMIC- CXR dataset. This illustrates the benefit of leveraging the chest imaging modality in the phenotyping task and highlights the potential of multi-modal learning in medical applications.
• Score: 3.11179491890629
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The healthcare domain is characterized by heterogeneous data modalities, such as imaging and physiological data. In practice, the variety of medical data assists clinicians in decision-making. However, most of the current state-of-the-art deep learning models solely rely upon carefully curated data of a single modality. In this paper, we propose a dynamic training approach to learn modality-specific data representations and to integrate auxiliary features, instead of solely relying on a single modality. Our preliminary experiments results for a patient phenotyping task using physiological data in MIMIC-IV & chest radiographs in the MIMIC- CXR dataset show that our proposed approach achieves the highest area under the receiver operating characteristic curve (AUROC) (0.764 AUROC) compared to the performance of the benchmark method in previous work, which only used physiological data (0.740 AUROC). For a set of five recurring or chronic diseases with periodic acute episodes, including cardiac dysrhythmia, conduction disorders, and congestive heart failure, the AUROC improves from 0.747 to 0.798. This illustrates the benefit of leveraging the chest imaging modality in the phenotyping task and highlights the potential of multi-modal learning in medical applications.

Related papers

• MDS-ED: Multimodal Decision Support in the Emergency Department -- a Benchmark Dataset for Diagnoses and Deterioration Prediction in Emergency Medicine [1.068128849363198]
  We introduce a dataset based on MIMIC-IV, benchmarking protocol, and initial results for evaluating multimodal decision support in the emergency department. We use diverse data modalities from the first 1.5 hours of patient arrival, including demographics, biometrics, vital signs, lab values, and electrocardiogram waveforms. Our diagnostic model achieves an AUROC score over 0.8 in a statistically significant manner for 357 out of 1428 conditions.
  arXiv  Detail & Related papers  (2024-07-25T08:21:46Z)
• Debiasing Cardiac Imaging with Controlled Latent Diffusion Models [1.802269171647208]
  We propose a method to alleviate imbalances inherent in datasets through the generation of synthetic data. We adopt ControlNet based on a denoising diffusion probabilistic model to condition on text assembled from patient metadata and cardiac geometry. Our experiments demonstrate the effectiveness of the proposed approach in mitigating dataset imbalances.
  arXiv  Detail & Related papers  (2024-03-28T15:41:43Z)
• Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation [113.5002649181103]
  Training open-source small multimodal models (SMMs) to bridge competency gaps for unmet clinical needs in radiology. For training, we assemble a large dataset of over 697 thousand radiology image-text pairs. For evaluation, we propose CheXprompt, a GPT-4-based metric for factuality evaluation, and demonstrate its parity with expert evaluation. The inference of LlaVA-Rad is fast and can be performed on a single V100 GPU in private settings, offering a promising state-of-the-art tool for real-world clinical applications.
  arXiv  Detail & Related papers  (2024-03-12T18:12:02Z)
• Radiology Report Generation Using Transformers Conditioned with Non-imaging Data [55.17268696112258]
  This paper proposes a novel multi-modal transformer network that integrates chest x-ray (CXR) images and associated patient demographic information. The proposed network uses a convolutional neural network to extract visual features from CXRs and a transformer-based encoder-decoder network that combines the visual features with semantic text embeddings of patient demographic information.
  arXiv  Detail & Related papers  (2023-11-18T14:52:26Z)
• Deep Learning for Predicting Progression of Patellofemoral Osteoarthritis Based on Lateral Knee Radiographs, Demographic Data and Symptomatic Assessments [1.1549572298362785]
  This study included subjects (1832 subjects, 3276 knees) from the baseline of the MOST study. PF joint regions-of-interest were identified using an automated landmark detection tool (BoneFinder) on lateral knee X-rays. Risk factors included age, sex, BMI and WOMAC score, and the radiographic osteoarthritis stage of the tibiofemoral joint (KL score)
  arXiv  Detail & Related papers  (2023-05-10T06:43:33Z)
• Tissue Classification During Needle Insertion Using Self-Supervised Contrastive Learning and Optical Coherence Tomography [53.38589633687604]
  We propose a deep neural network that classifies the tissues from the phase and intensity data of complex OCT signals acquired at the needle tip. We show that with 10% of the training set, our proposed pretraining strategy helps the model achieve an F1 score of 0.84 whereas the model achieves an F1 score of 0.60 without it.
  arXiv  Detail & Related papers  (2023-04-26T14:11:04Z)
• Bootstrapping Your Own Positive Sample: Contrastive Learning With Electronic Health Record Data [62.29031007761901]
  This paper proposes a novel contrastive regularized clinical classification model. We introduce two unique positive sampling strategies specifically tailored for EHR data. Our framework yields highly competitive experimental results in predicting the mortality risk on real-world COVID-19 EHR data.
  arXiv  Detail & Related papers  (2021-04-07T06:02:04Z)
• Longitudinal modeling of MS patient trajectories improves predictions of disability progression [2.117653457384462]
  This work addresses the task of optimally extracting information from longitudinal patient data in the real-world setting. We show that with machine learning methods suited for patient trajectories modeling, we can predict disability progression of patients in a two-year horizon. Compared to the models available in the literature, this work uses the most complete patient history for MS disease progression prediction.
  arXiv  Detail & Related papers  (2020-11-09T20:48:00Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Trajectories, bifurcations and pseudotime in large clinical datasets: applications to myocardial infarction and diabetes data [94.37521840642141]
  We suggest a semi-supervised methodology for the analysis of large clinical datasets, characterized by mixed data types and missing values. The methodology is based on application of elastic principal graphs which can address simultaneously the tasks of dimensionality reduction, data visualization, clustering, feature selection and quantifying the geodesic distances (pseudotime) in partially ordered sequences of observations.
  arXiv  Detail & Related papers  (2020-07-07T21:04:55Z)
• Prediction of Thrombectomy Functional Outcomes using Multimodal Data [2.358784542343728]
  We propose a novel deep learning approach to directly exploit multimodal data to estimate the success of endovascular treatment. We incorporate an attention mechanism in our architecture to model global feature inter-dependencies, both channel-wise and spatially.
  arXiv  Detail & Related papers  (2020-05-26T21:51:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.