CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting

• URL: http://arxiv.org/abs/2002.11379v2
• Date: Wed, 11 Mar 2020 07:15:57 GMT
• Title: CheXpedition: Investigating Generalization Challenges for Translation of Chest X-Ray Algorithms to the Clinical Setting
• Authors: Pranav Rajpurkar, Anirudh Joshi, Anuj Pareek, Phil Chen, Amirhossein Kiani, Jeremy Irvin, Andrew Y. Ng, Matthew P. Lungren
• Abstract summary: We examine the performance of the top 10 performing models on the CheXpert challenge leaderboard. We find that the top 10 chest x-ray models achieve an average AUC of 0.851 on the task of detecting TB on two public TB datasets. Second, we find that the average performance of the models on photos of x-rays (AUC = 0.916) is similar to their performance on the original chest x-ray images. Third, we find that the models tested on an external dataset either perform comparably to or exceed the average performance of radiologists.
• Score: 4.781964929315763
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Although there have been several recent advances in the application of deep learning algorithms to chest x-ray interpretation, we identify three major challenges for the translation of chest x-ray algorithms to the clinical setting. We examine the performance of the top 10 performing models on the CheXpert challenge leaderboard on three tasks: (1) TB detection, (2) pathology detection on photos of chest x-rays, and (3) pathology detection on data from an external institution. First, we find that the top 10 chest x-ray models on the CheXpert competition achieve an average AUC of 0.851 on the task of detecting TB on two public TB datasets without fine-tuning or including the TB labels in training data. Second, we find that the average performance of the models on photos of x-rays (AUC = 0.916) is similar to their performance on the original chest x-ray images (AUC = 0.924). Third, we find that the models tested on an external dataset either perform comparably to or exceed the average performance of radiologists. We believe that our investigation will inform rapid translation of deep learning algorithms to safe and effective clinical decision support tools that can be validated prospectively with large impact studies and clinical trials.

Related papers

• Robust and Interpretable COVID-19 Diagnosis on Chest X-ray Images using Adversarial Training [0.8287206589886881]
  We train 21 convolutional neural network (CNN) models on a diverse set of 33,000+ chest X-ray (CXR) images to classify between healthy, COVID-19, and non-COVID-19 pneumonia CXRs. Our resulting models achieved a 3-way classification accuracy, recall, and precision of up to 97.03%, 97.97%, and 99.95%, respectively.
  arXiv  Detail & Related papers  (2023-11-23T23:40:01Z)
• Revisiting Computer-Aided Tuberculosis Diagnosis [56.80999479735375]
  Tuberculosis (TB) is a major global health threat, causing millions of deaths annually. Computer-aided tuberculosis diagnosis (CTD) using deep learning has shown promise, but progress is hindered by limited training data. We establish a large-scale dataset, namely the Tuberculosis X-ray (TBX11K) dataset, which contains 11,200 chest X-ray (CXR) images with corresponding bounding box annotations for TB areas. This dataset enables the training of sophisticated detectors for high-quality CTD.
  arXiv  Detail & Related papers  (2023-07-06T08:27:48Z)
• SwinCheX: Multi-label classification on chest X-ray images with transformers [4.549831511476249]
  This paper proposes a multi-label classification deep model based on the Swin Transformer as the backbone to achieve state-of-the-art diagnosis classification. We evaluate our model on one of the most widely-used and largest x-ray datasets called "Chest X-ray14"
  arXiv  Detail & Related papers  (2022-06-09T03:17:57Z)
• The pitfalls of using open data to develop deep learning solutions for COVID-19 detection in chest X-rays [64.02097860085202]
  Deep learning models have been developed to identify COVID-19 from chest X-rays. Results have been exceptional when training and testing on open-source data. Data analysis and model evaluations show that the popular open-source dataset COVIDx is not representative of the real clinical problem.
  arXiv  Detail & Related papers  (2021-09-14T10:59:11Z)
• CheXbreak: Misclassification Identification for Deep Learning Models Interpreting Chest X-rays [5.263502842508203]
  We first investigate whether there are patient subgroups that chest x-ray models are likely to misclassify. Patient age and the radiographic finding of lung lesion or pneumothorax are statistically relevant features for predicting misclassification for some chest x-ray models. We develop misclassification predictors on chest x-ray models using their outputs and clinical features.
  arXiv  Detail & Related papers  (2021-03-18T00:30:19Z)
• CheXternal: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays and External Clinical Settings [6.133159722996137]
  We measured the diagnostic performance for 8 different chest X-ray models when applied to smartphone photos of chest X-rays and external datasets. We found that on photos of chest X-rays, all 8 models experienced a statistically significant drop in task performance. Some chest X-ray models, under clinically relevant distribution shifts, were comparable to radiologists while other models were not.
  arXiv  Detail & Related papers  (2021-02-17T09:58:14Z)
• Pristine annotations-based multi-modal trained artificial intelligence solution to triage chest X-ray for COVID-19 [1.1764495014312295]
  COVID-19 pandemic continues to spread and impact the well-being of the global population. Front-line modalities including computed tomography (CT) and X-ray play an important role for triaging COVID patients. Considering the limited access of resources (both hardware and trained personnel) and decontamination considerations, CT may not be ideal for triaging suspected subjects.
  arXiv  Detail & Related papers  (2020-11-10T15:36:08Z)
• Contralaterally Enhanced Networks for Thoracic Disease Detection [120.60868136876599]
  There exist many similar structures in the left and right parts of the chest, such as ribs, lung fields and bronchial tubes. This kind of similarities can be used to identify diseases in chest X-rays, according to the experience of broad-certificated radiologists. We propose a deep end-to-end module to exploit the contralateral context information for enhancing feature representations of disease proposals.
  arXiv  Detail & Related papers  (2020-10-09T10:15:26Z)
• Learning Invariant Feature Representation to Improve Generalization across Chest X-ray Datasets [55.06983249986729]
  We show that a deep learning model performing well when tested on the same dataset as training data starts to perform poorly when it is tested on a dataset from a different source. By employing an adversarial training strategy, we show that a network can be forced to learn a source-invariant representation.
  arXiv  Detail & Related papers  (2020-08-04T07:41:15Z)
• Deep Mining External Imperfect Data for Chest X-ray Disease Screening [57.40329813850719]
  We argue that incorporating an external CXR dataset leads to imperfect training data, which raises the challenges. We formulate the multi-label disease classification problem as weighted independent binary tasks according to the categories. Our framework simultaneously models and tackles the domain and label discrepancies, enabling superior knowledge mining ability.
  arXiv  Detail & Related papers  (2020-06-06T06:48:40Z)
• Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning [57.00601760750389]
  We present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images. Such a tool can gauge severity of COVID-19 lung infections that can be used for escalation or de-escalation of care.
  arXiv  Detail & Related papers  (2020-05-24T23:13:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.