Was there COVID-19 back in 2012? Challenge for AI in Diagnosis with Similar Indications

• URL: http://arxiv.org/abs/2006.13262v1
• Date: Tue, 23 Jun 2020 18:35:57 GMT
• Title: Was there COVID-19 back in 2012? Challenge for AI in Diagnosis with Similar Indications
• Authors: Imon Banerjee, Priyanshu Sinha, Saptarshi Purkayastha, Nazanin Mashhaditafreshi, Amara Tariq, Jiwoong Jeong, Hari Trivedi, Judy W. Gichoya
• Abstract summary: Since the recent COVID-19 outbreak, there has been an avalanche of research papers applying deep learning based image processing to chest radiographs for detection of the disease. We present our argument regarding the efficiency and applicability of existing deep learning models for COVID-19 diagnosis.
• Score: 2.250064549069322
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose: Since the recent COVID-19 outbreak, there has been an avalanche of research papers applying deep learning based image processing to chest radiographs for detection of the disease. To test the performance of the two top models for CXR COVID-19 diagnosis on external datasets to assess model generalizability. Methods: In this paper, we present our argument regarding the efficiency and applicability of existing deep learning models for COVID-19 diagnosis. We provide results from two popular models - COVID-Net and CoroNet evaluated on three publicly available datasets and an additional institutional dataset collected from EMORY Hospital between January and May 2020, containing patients tested for COVID-19 infection using RT-PCR. Results: There is a large false positive rate (FPR) for COVID-Net on both ChexPert (55.3%) and MIMIC-CXR (23.4%) dataset. On the EMORY Dataset, COVID-Net has 61.4% sensitivity, 0.54 F1-score and 0.49 precision value. The FPR of the CoroNet model is significantly lower across all the datasets as compared to COVID-Net - EMORY(9.1%), ChexPert (1.3%), ChestX-ray14 (0.02%), MIMIC-CXR (0.06%). Conclusion: The models reported good to excellent performance on their internal datasets, however we observed from our testing that their performance dramatically worsened on external data. This is likely from several causes including overfitting models due to lack of appropriate control patients and ground truth labels. The fourth institutional dataset was labeled using RT-PCR, which could be positive without radiographic findings and vice versa. Therefore, a fusion model of both clinical and radiographic data may have better performance and generalization.

Related papers

• COVIDx CXR-4: An Expanded Multi-Institutional Open-Source Benchmark Dataset for Chest X-ray Image-Based Computer-Aided COVID-19 Diagnostics [79.90346960083775]
  We introduce COVIDx CXR-4, an expanded multi-institutional open-source benchmark dataset for chest X-ray image-based computer-aided COVID-19 diagnostics. COVIDx CXR-4 expands significantly on the previous COVIDx CXR-3 dataset by increasing the total patient cohort size by greater than 2.66 times. We provide extensive analysis on the diversity of the patient demographic, imaging metadata, and disease distributions to highlight potential dataset biases.
  arXiv  Detail & Related papers  (2023-11-29T14:40:31Z)
• CIRCA: comprehensible online system in support of chest X-rays-based COVID-19 diagnosis [37.41181188499616]
  Deep learning techniques can help in the faster detection of COVID-19 cases and monitoring of disease progression. Five different datasets were used to construct a representative dataset of 23 799 CXRs for model training. A U-Net-based model was developed to identify a clinically relevant region of the CXR.
  arXiv  Detail & Related papers  (2022-10-11T13:30:34Z)
• A Generalizable Artificial Intelligence Model for COVID-19 Classification Task Using Chest X-ray Radiographs: Evaluated Over Four Clinical Datasets with 15,097 Patients [6.209420804714487]
  The generalizability of the trained model was retrospectively evaluated using four different real-world clinical datasets. The AI model trained using a single-source clinical dataset achieved an AUC of 0.82 when applied to the internal temporal test set. An AUC of 0.79 was achieved when applied to a multi-institutional COVID-19 dataset collected by the Medical Imaging and Data Resource Center.
  arXiv  Detail & Related papers  (2022-10-04T04:12:13Z)
• The Report on China-Spain Joint Clinical Testing for Rapid COVID-19 Risk Screening by Eye-region Manifestations [59.48245489413308]
  We developed and tested a COVID-19 rapid prescreening model using the eye-region images captured in China and Spain with cellphone cameras. The performance was measured using area under receiver-operating-characteristic curve (AUC), sensitivity, specificity, accuracy, and F1.
  arXiv  Detail & Related papers  (2021-09-18T02:28:01Z)
• The pitfalls of using open data to develop deep learning solutions for COVID-19 detection in chest X-rays [64.02097860085202]
  Deep learning models have been developed to identify COVID-19 from chest X-rays. Results have been exceptional when training and testing on open-source data. Data analysis and model evaluations show that the popular open-source dataset COVIDx is not representative of the real clinical problem.
  arXiv  Detail & Related papers  (2021-09-14T10:59:11Z)
• COVID-Net CXR-2: An Enhanced Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-ray Images [58.35627258364233]
  Use of chest X-ray (CXR) imaging as a complimentary screening strategy to RT-PCR testing continues to grow. We introduce COVID-Net CXR-2, an enhanced deep convolutional neural network design for COVID-19 detection from CXR images. benchmark dataset composed of 19,203 CXR images from a multinational cohort of 16,656 patients from at least 51 countries.
  arXiv  Detail & Related papers  (2021-05-14T04:29:21Z)
• Systematic investigation into generalization of COVID-19 CT deep learning models with Gabor ensemble for lung involvement scoring [9.94980188821453]
  This study investigates the generalizability of key published models using the publicly available COVID-19 Computed Tomography data. We then assess the predictive ability of these models for COVID-19 severity using an independent new dataset.
  arXiv  Detail & Related papers  (2021-04-20T03:49:48Z)
• Deep learning-based COVID-19 pneumonia classification using chest CT images: model generalizability [54.86482395312936]
  Deep learning (DL) classification models were trained to identify COVID-19-positive patients on 3D computed tomography (CT) datasets from different countries. We trained nine identical DL-based classification models by using combinations of the datasets with a 72% train, 8% validation, and 20% test data split. The models trained on multiple datasets and evaluated on a test set from one of the datasets used for training performed better.
  arXiv  Detail & Related papers  (2021-02-18T21:14:52Z)
• COVID-19 Classification of X-ray Images Using Deep Neural Networks [36.99143569437537]
  The purpose of this study is to create and evaluate a machine learning model for diagnosis of COVID-19. A machine learning model was built using a pre-trained deep learning model (ReNet50) and enhanced by data augmentation and lung segmentation. The model was evaluated using accuracy, sensitivity, area under the curve (AUC) of receiver operating characteristic (ROC) curve and of the precision-recall (P-R) curve.
  arXiv  Detail & Related papers  (2020-10-03T13:57:08Z)
• COVIDGR dataset and COVID-SDNet methodology for predicting COVID-19 based on Chest X-Ray images [0.0]
  Currently, Coronavirus disease (COVID-19) is diagnosed using RT-PCR testing, CT scans and/or Chest X-Ray (CXR) images. Deep learning neural networks have a great potential for building COVID-19 triage systems. This paper is three-fold: (i) we demystify the high sensitivities achieved by most recent COVID-19 classification models, (ii) under a close collaboration with Hospital Universitario Cl'inico San Cecilio, Granada, Spain, we built COVIDGR-1.0, and (iii) we propose COVID Smart Data based Network (COVID-SD
  arXiv  Detail & Related papers  (2020-06-02T06:18:34Z)
• Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning [57.00601760750389]
  We present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images. Such a tool can gauge severity of COVID-19 lung infections that can be used for escalation or de-escalation of care.
  arXiv  Detail & Related papers  (2020-05-24T23:13:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.