The pitfalls of using open data to develop deep learning solutions for COVID-19 detection in chest X-rays

• URL: http://arxiv.org/abs/2109.08020v1
• Date: Tue, 14 Sep 2021 10:59:11 GMT
• Title: The pitfalls of using open data to develop deep learning solutions for COVID-19 detection in chest X-rays
• Authors: Rachael Harkness, Geoff Hall, Alejandro F Frangi, Nishant Ravikumar, Kieran Zucker
• Abstract summary: Deep learning models have been developed to identify COVID-19 from chest X-rays. Results have been exceptional when training and testing on open-source data. Data analysis and model evaluations show that the popular open-source dataset COVIDx is not representative of the real clinical problem.
• Score: 64.02097860085202
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Since the emergence of COVID-19, deep learning models have been developed to identify COVID-19 from chest X-rays. With little to no direct access to hospital data, the AI community relies heavily on public data comprising numerous data sources. Model performance results have been exceptional when training and testing on open-source data, surpassing the reported capabilities of AI in pneumonia-detection prior to the COVID-19 outbreak. In this study impactful models are trained on a widely used open-source data and tested on an external test set and a hospital dataset, for the task of classifying chest X-rays into one of three classes: COVID-19, non-COVID pneumonia and no-pneumonia. Classification performance of the models investigated is evaluated through ROC curves, confusion matrices and standard classification metrics. Explainability modules are implemented to explore the image features most important to classification. Data analysis and model evaluations show that the popular open-source dataset COVIDx is not representative of the real clinical problem and that results from testing on this are inflated. Dependence on open-source data can leave models vulnerable to bias and confounding variables, requiring careful analysis to develop clinically useful/viable AI tools for COVID-19 detection in chest X-rays.

Related papers

• Secure Federated Learning Approaches to Diagnosing COVID-19 [0.0]
  This paper introduces a HIPAA-compliant model to aid in the diagnosis of COVID-19. Federated learning is a distributed machine learning approach that allows for algorithm training across multiple decentralized devices. To enhance hospital understanding of the model, we employed a visualization technique that highlights key features in chest X-rays indicative of a positive COVID-19 diagnosis.
  arXiv  Detail & Related papers  (2024-01-23T02:14:05Z)
• Deep Reinforcement Learning Framework for Thoracic Diseases Classification via Prior Knowledge Guidance [49.87607548975686]
  The scarcity of labeled data for related diseases poses a huge challenge to an accurate diagnosis. We propose a novel deep reinforcement learning framework, which introduces prior knowledge to direct the learning of diagnostic agents. Our approach's performance was demonstrated using the well-known NIHX-ray 14 and CheXpert datasets.
  arXiv  Detail & Related papers  (2023-06-02T01:46:31Z)
• Challenges in COVID-19 Chest X-Ray Classification: Problematic Data or Ineffective Approaches? [0.0]
  deep learning to classify and detect COVID-19 infections from chest radiography images. In this work, we investigate the challenges faced with creating reliable AI solutions from both the data and machine learning perspectives.
  arXiv  Detail & Related papers  (2022-01-16T14:12:04Z)
• DPCOVID: Privacy-Preserving Federated Covid-19 Detection [2.1930130356902207]
  Coronavirus (COVID-19) has shown an unprecedented global crisis by the detrimental effect on the global economy and health. We present a privacy-preserving Federated Learning system for COVID-19 detection based on chest X-ray images.
  arXiv  Detail & Related papers  (2021-10-26T15:09:00Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• RANDGAN: Randomized Generative Adversarial Network for Detection of COVID-19 in Chest X-ray [0.0]
  COVID-19 spread across the globe at an immense rate has left healthcare systems incapacitated to diagnose and test patients at the needed rate. Studies have shown promising results for detection of COVID-19 from viral bacterial pneumonia in chest X-rays. In this study, we propose a randomized generative adversarial network (RANDGAN) that detects images of an unknown class (COVID-19) without the need for labels and training data.
  arXiv  Detail & Related papers  (2020-10-06T15:58:09Z)
• Learning Invariant Feature Representation to Improve Generalization across Chest X-ray Datasets [55.06983249986729]
  We show that a deep learning model performing well when tested on the same dataset as training data starts to perform poorly when it is tested on a dataset from a different source. By employing an adversarial training strategy, we show that a network can be forced to learn a source-invariant representation.
  arXiv  Detail & Related papers  (2020-08-04T07:41:15Z)
• Deep Mining External Imperfect Data for Chest X-ray Disease Screening [57.40329813850719]
  We argue that incorporating an external CXR dataset leads to imperfect training data, which raises the challenges. We formulate the multi-label disease classification problem as weighted independent binary tasks according to the categories. Our framework simultaneously models and tackles the domain and label discrepancies, enabling superior knowledge mining ability.
  arXiv  Detail & Related papers  (2020-06-06T06:48:40Z)
• Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning [57.00601760750389]
  We present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images. Such a tool can gauge severity of COVID-19 lung infections that can be used for escalation or de-escalation of care.
  arXiv  Detail & Related papers  (2020-05-24T23:13:16Z)
• Unveiling COVID-19 from Chest X-ray with deep learning: a hurdles race with small data [8.253442222027134]
  The possibility to use widespread and simple chest X-ray (CXR) imaging for early screening of COVID-19 patients is attracting much interest. We provide insights and raise warnings on what is reasonable to expect by applying deep-learning to COVID classification of CXR images.
  arXiv  Detail & Related papers  (2020-04-11T13:58:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.