A Generalizable Artificial Intelligence Model for COVID-19 Classification Task Using Chest X-ray Radiographs: Evaluated Over Four Clinical Datasets with 15,097 Patients

• URL: http://arxiv.org/abs/2210.02189v1
• Date: Tue, 4 Oct 2022 04:12:13 GMT
• Title: A Generalizable Artificial Intelligence Model for COVID-19 Classification Task Using Chest X-ray Radiographs: Evaluated Over Four Clinical Datasets with 15,097 Patients
• Authors: Ran Zhang, Xin Tie, John W. Garrett, Dalton Griner, Zhihua Qi, Nicholas B. Bevins, Scott B. Reeder and Guang-Hong Chen
• Abstract summary: The generalizability of the trained model was retrospectively evaluated using four different real-world clinical datasets. The AI model trained using a single-source clinical dataset achieved an AUC of 0.82 when applied to the internal temporal test set. An AUC of 0.79 was achieved when applied to a multi-institutional COVID-19 dataset collected by the Medical Imaging and Data Resource Center.
• Score: 6.209420804714487
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: To answer the long-standing question of whether a model trained from a single clinical site can be generalized to external sites. Materials and Methods: 17,537 chest x-ray radiographs (CXRs) from 3,264 COVID-19-positive patients and 4,802 COVID-19-negative patients were collected from a single site for AI model development. The generalizability of the trained model was retrospectively evaluated using four different real-world clinical datasets with a total of 26,633 CXRs from 15,097 patients (3,277 COVID-19-positive patients). The area under the receiver operating characteristic curve (AUC) was used to assess diagnostic performance. Results: The AI model trained using a single-source clinical dataset achieved an AUC of 0.82 (95% CI: 0.80, 0.84) when applied to the internal temporal test set. When applied to datasets from two external clinical sites, an AUC of 0.81 (95% CI: 0.80, 0.82) and 0.82 (95% CI: 0.80, 0.84) were achieved. An AUC of 0.79 (95% CI: 0.77, 0.81) was achieved when applied to a multi-institutional COVID-19 dataset collected by the Medical Imaging and Data Resource Center (MIDRC). A power-law dependence, N^(k )(k is empirically found to be -0.21 to -0.25), indicates a relatively weak performance dependence on the training data sizes. Conclusion: COVID-19 classification AI model trained using well-curated data from a single clinical site is generalizable to external clinical sites without a significant drop in performance.

Related papers

• Incorporating Anatomical Awareness for Enhanced Generalizability and Progression Prediction in Deep Learning-Based Radiographic Sacroiliitis Detection [0.8248058061511542]
  The aim of this study was to examine whether incorporating anatomical awareness into a deep learning model can improve generalizability and enable prediction of disease progression. The performance of the models was compared using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.
  arXiv  Detail & Related papers  (2024-05-12T20:02:25Z)
• Detection of subclinical atherosclerosis by image-based deep learning on chest x-ray [86.38767955626179]
  Deep-learning algorithm to predict coronary artery calcium (CAC) score was developed on 460 chest x-ray. The diagnostic accuracy of the AICAC model assessed by the area under the curve (AUC) was the primary outcome.
  arXiv  Detail & Related papers  (2024-03-27T16:56:14Z)
• DeepCOVID-Fuse: A Multi-modality Deep Learning Model Fusing Chest X-Radiographs and Clinical Variables to Predict COVID-19 Risk Levels [8.593516170110203]
  DeepCOVID-Fuse is a deep learning fusion model to predict risk levels in coronavirus patients. The accuracy of DeepCOVID-Fuse trained on CXRs and clinical variables is 0.658, with an AUC of 0.842.
  arXiv  Detail & Related papers  (2023-01-20T20:54:25Z)
• Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence [79.038671794961]
  We launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK.
  arXiv  Detail & Related papers  (2021-11-18T00:43:41Z)
• The Report on China-Spain Joint Clinical Testing for Rapid COVID-19 Risk Screening by Eye-region Manifestations [59.48245489413308]
  We developed and tested a COVID-19 rapid prescreening model using the eye-region images captured in China and Spain with cellphone cameras. The performance was measured using area under receiver-operating-characteristic curve (AUC), sensitivity, specificity, accuracy, and F1.
  arXiv  Detail & Related papers  (2021-09-18T02:28:01Z)
• The pitfalls of using open data to develop deep learning solutions for COVID-19 detection in chest X-rays [64.02097860085202]
  Deep learning models have been developed to identify COVID-19 from chest X-rays. Results have been exceptional when training and testing on open-source data. Data analysis and model evaluations show that the popular open-source dataset COVIDx is not representative of the real clinical problem.
  arXiv  Detail & Related papers  (2021-09-14T10:59:11Z)
• Deep learning-based COVID-19 pneumonia classification using chest CT images: model generalizability [54.86482395312936]
  Deep learning (DL) classification models were trained to identify COVID-19-positive patients on 3D computed tomography (CT) datasets from different countries. We trained nine identical DL-based classification models by using combinations of the datasets with a 72% train, 8% validation, and 20% test data split. The models trained on multiple datasets and evaluated on a test set from one of the datasets used for training performed better.
  arXiv  Detail & Related papers  (2021-02-18T21:14:52Z)
• Chest x-ray automated triage: a semiologic approach designed for clinical implementation, exploiting different types of labels through a combination of four Deep Learning architectures [83.48996461770017]
  This work presents a Deep Learning method based on the late fusion of different convolutional architectures. We built four training datasets combining images from public chest x-ray datasets and our institutional archive. We trained four different Deep Learning architectures and combined their outputs with a late fusion strategy, obtaining a unified tool.
  arXiv  Detail & Related papers  (2020-12-23T14:38:35Z)
• COVID-19 Classification of X-ray Images Using Deep Neural Networks [36.99143569437537]
  The purpose of this study is to create and evaluate a machine learning model for diagnosis of COVID-19. A machine learning model was built using a pre-trained deep learning model (ReNet50) and enhanced by data augmentation and lung segmentation. The model was evaluated using accuracy, sensitivity, area under the curve (AUC) of receiver operating characteristic (ROC) curve and of the precision-recall (P-R) curve.
  arXiv  Detail & Related papers  (2020-10-03T13:57:08Z)
• Was there COVID-19 back in 2012? Challenge for AI in Diagnosis with Similar Indications [2.250064549069322]
  Since the recent COVID-19 outbreak, there has been an avalanche of research papers applying deep learning based image processing to chest radiographs for detection of the disease. We present our argument regarding the efficiency and applicability of existing deep learning models for COVID-19 diagnosis.
  arXiv  Detail & Related papers  (2020-06-23T18:35:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.