CheXbreak: Misclassification Identification for Deep Learning Models Interpreting Chest X-rays

• URL: http://arxiv.org/abs/2103.09957v1
• Date: Thu, 18 Mar 2021 00:30:19 GMT
• Title: CheXbreak: Misclassification Identification for Deep Learning Models Interpreting Chest X-rays
• Authors: Emma Chen, Andy Kim, Rayan Krishnan, Jin Long, Andrew Y. Ng, Pranav Rajpurkar
• Abstract summary: We first investigate whether there are patient subgroups that chest x-ray models are likely to misclassify. Patient age and the radiographic finding of lung lesion or pneumothorax are statistically relevant features for predicting misclassification for some chest x-ray models. We develop misclassification predictors on chest x-ray models using their outputs and clinical features.
• Score: 5.263502842508203
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A major obstacle to the integration of deep learning models for chest x-ray interpretation into clinical settings is the lack of understanding of their failure modes. In this work, we first investigate whether there are patient subgroups that chest x-ray models are likely to misclassify. We find that patient age and the radiographic finding of lung lesion or pneumothorax are statistically relevant features for predicting misclassification for some chest x-ray models. Second, we develop misclassification predictors on chest x-ray models using their outputs and clinical features. We find that our best performing misclassification identifier achieves an AUROC close to 0.9 for most diseases. Third, employing our misclassification identifiers, we develop a corrective algorithm to selectively flip model predictions that have high likelihood of misclassification at inference time. We observe F1 improvement on the prediction of Consolidation (0.008 [95\% CI 0.005, 0.010]) and Edema (0.003, [95\% CI 0.001, 0.006]). By carrying out our investigation on ten distinct and high-performing chest x-ray models, we are able to derive insights across model architectures and offer a generalizable framework applicable to other medical imaging tasks.

Related papers

• SynthEnsemble: A Fusion of CNN, Vision Transformer, and Hybrid Models for Multi-Label Chest X-Ray Classification [0.6218519716921521]
  We employ deep learning techniques to identify patterns in chest X-rays that correspond to different diseases. The best individual model was the CoAtNet, which achieved an area under the receiver operating characteristic curve (AUROC) of 84.2%.
  arXiv  Detail & Related papers  (2023-11-13T21:07:07Z)
• CheX-Nomaly: Segmenting Lung Abnormalities from Chest Radiographs using Machine Learning [0.0]
  We present CheX-nomaly: a binary localization U-net model with the incorporation of an innovative contrastive learning approach. We show that we can significantly improve the generalizability of an abnormality localization model by incorporating a contrastive learning method. We also introduce a new loss technique to apply to enhance the U-nets performance on bounding box segmentation.
  arXiv  Detail & Related papers  (2023-11-03T08:27:57Z)
• Deep Reinforcement Learning Framework for Thoracic Diseases Classification via Prior Knowledge Guidance [49.87607548975686]
  The scarcity of labeled data for related diseases poses a huge challenge to an accurate diagnosis. We propose a novel deep reinforcement learning framework, which introduces prior knowledge to direct the learning of diagnostic agents. Our approach's performance was demonstrated using the well-known NIHX-ray 14 and CheXpert datasets.
  arXiv  Detail & Related papers  (2023-06-02T01:46:31Z)
• ThoraX-PriorNet: A Novel Attention-Based Architecture Using Anatomical Prior Probability Maps for Thoracic Disease Classification [2.0319363307774476]
  It is known that different thoracic disease lesions are more likely to occur in specific anatomical regions compared to others. This article aims to incorporate this disease and region-dependent prior probability distribution within a deep learning framework.
  arXiv  Detail & Related papers  (2022-10-06T15:38:02Z)
• A Deep Learning Technique using a Sequence of Follow Up X-Rays for Disease classification [3.3345134768053635]
  The ability to predict lung and heart based diseases using deep learning techniques is central to many researchers. We present a hypothesis that X-rays of patients included with the follow up history of their most recent three chest X-ray images would perform better in disease classification.
  arXiv  Detail & Related papers  (2022-03-28T19:58:47Z)
• The pitfalls of using open data to develop deep learning solutions for COVID-19 detection in chest X-rays [64.02097860085202]
  Deep learning models have been developed to identify COVID-19 from chest X-rays. Results have been exceptional when training and testing on open-source data. Data analysis and model evaluations show that the popular open-source dataset COVIDx is not representative of the real clinical problem.
  arXiv  Detail & Related papers  (2021-09-14T10:59:11Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• CheXternal: Generalization of Deep Learning Models for Chest X-ray Interpretation to Photos of Chest X-rays and External Clinical Settings [6.133159722996137]
  We measured the diagnostic performance for 8 different chest X-ray models when applied to smartphone photos of chest X-rays and external datasets. We found that on photos of chest X-rays, all 8 models experienced a statistically significant drop in task performance. Some chest X-ray models, under clinically relevant distribution shifts, were comparable to radiologists while other models were not.
  arXiv  Detail & Related papers  (2021-02-17T09:58:14Z)
• Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning [57.00601760750389]
  We present a severity score prediction model for COVID-19 pneumonia for frontal chest X-ray images. Such a tool can gauge severity of COVID-19 lung infections that can be used for escalation or de-escalation of care.
  arXiv  Detail & Related papers  (2020-05-24T23:13:16Z)
• Self-Training with Improved Regularization for Sample-Efficient Chest X-Ray Classification [80.00316465793702]
  We present a deep learning framework that enables robust modeling in challenging scenarios. Our results show that using 85% lesser labeled data, we can build predictive models that match the performance of classifiers trained in a large-scale data setting.
  arXiv  Detail & Related papers  (2020-05-03T02:36:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.