Rethinking annotation granularity for overcoming deep shortcut learning: A retrospective study on chest radiographs

• URL: http://arxiv.org/abs/2104.10553v1
• Date: Wed, 21 Apr 2021 14:21:37 GMT
• Title: Rethinking annotation granularity for overcoming deep shortcut learning: A retrospective study on chest radiographs
• Authors: Luyang Luo, Hao Chen, Yongjie Xiao, Yanning Zhou, Xi Wang, Varut Vardhanabhuti, Mingxiang Wu, Pheng-Ann Heng
• Abstract summary: We compare a popular thoracic disease classification model, CheXNet, and a thoracic lesion detection model, CheXDet. We found incorporating external training data even led to performance degradation for CheXNet. By visualizing the models' decision-making regions, we revealed that CheXNet learned patterns other than the target lesions.
• Score: 43.43732218093039
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Deep learning has demonstrated radiograph screening performances that are comparable or superior to radiologists. However, recent studies show that deep models for thoracic disease classification usually show degraded performance when applied to external data. Such phenomena can be categorized into shortcut learning, where the deep models learn unintended decision rules that can fit the identically distributed training and test set but fail to generalize to other distributions. A natural way to alleviate this defect is explicitly indicating the lesions and focusing the model on learning the intended features. In this paper, we conduct extensive retrospective experiments to compare a popular thoracic disease classification model, CheXNet, and a thoracic lesion detection model, CheXDet. We first showed that the two models achieved similar image-level classification performance on the internal test set with no significant differences under many scenarios. Meanwhile, we found incorporating external training data even led to performance degradation for CheXNet. Then, we compared the models' internal performance on the lesion localization task and showed that CheXDet achieved significantly better performance than CheXNet even when given 80% less training data. By further visualizing the models' decision-making regions, we revealed that CheXNet learned patterns other than the target lesions, demonstrating its shortcut learning defect. Moreover, CheXDet achieved significantly better external performance than CheXNet on both the image-level classification task and the lesion localization task. Our findings suggest improving annotation granularity for training deep learning systems as a promising way to elevate future deep learning-based diagnosis systems for clinical usage.

Related papers

• Comparative study of Deep Learning Models for Binary Classification on Combined Pulmonary Chest X-ray Dataset [0.0]
  We compared the binary classification performance of eight prominent deep learning models: DenseNet 121, DenseNet 169, DenseNet 201, EffecientNet b0, EffecientNet lite4, GoogleNet, MobileNet, and ResNet18. We found a distinct difference in performance among the other models when applied to the pulmonary chest Xray image dataset.
  arXiv  Detail & Related papers  (2023-09-16T11:58:04Z)
• Performance of GAN-based augmentation for deep learning COVID-19 image classification [57.1795052451257]
  The biggest challenge in the application of deep learning to the medical domain is the availability of training data. Data augmentation is a typical methodology used in machine learning when confronted with a limited data set. In this work, a StyleGAN2-ADA model of Generative Adversarial Networks is trained on the limited COVID-19 chest X-ray image set.
  arXiv  Detail & Related papers  (2023-04-18T15:39:58Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• Interpretable COVID-19 Chest X-Ray Classification via Orthogonality Constraint [10.190872613479632]
  We investigate the benefit of using Orthogonal Spheres (OS) constraint for classification of COVID-19 cases from chest X-ray images. Previous studies have demonstrated significant benefits in applying such constraints to deep learning models. Our approach achieves an improvement in accuracy of 1.6% and 4.8% for two- and three-class classification.
  arXiv  Detail & Related papers  (2021-02-02T11:35:28Z)
• Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization [2.1699022621790736]
  We let six radiologists rate the interpretability of saliency maps in datasets of X-rays, computed tomography, and magnetic resonance imaging scans. We found that the accuracy of adversarially trained models was equal to standard models when sufficiently large datasets and dual batch norm training were used.
  arXiv  Detail & Related papers  (2020-11-25T20:41:01Z)
• Learning Invariant Feature Representation to Improve Generalization across Chest X-ray Datasets [55.06983249986729]
  We show that a deep learning model performing well when tested on the same dataset as training data starts to perform poorly when it is tested on a dataset from a different source. By employing an adversarial training strategy, we show that a network can be forced to learn a source-invariant representation.
  arXiv  Detail & Related papers  (2020-08-04T07:41:15Z)
• Deep Mining External Imperfect Data for Chest X-ray Disease Screening [57.40329813850719]
  We argue that incorporating an external CXR dataset leads to imperfect training data, which raises the challenges. We formulate the multi-label disease classification problem as weighted independent binary tasks according to the categories. Our framework simultaneously models and tackles the domain and label discrepancies, enabling superior knowledge mining ability.
  arXiv  Detail & Related papers  (2020-06-06T06:48:40Z)
• Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-rays [3.785818062712446]
  This disease is caused by the novel Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) virus. A custom convolutional neural network and a selection of ImageNet pretrained models are trained and evaluated at patient-level. The learned knowledge is transferred and fine-tuned to improve performance and generalization.
  arXiv  Detail & Related papers  (2020-04-16T00:09:29Z)
• An Adversarial Approach for the Robust Classification of Pneumonia from Chest Radiographs [9.462808515258464]
  Deep learning models often exhibit performance loss due to dataset shift. Models trained using data from one hospital system achieve high predictive performance when tested on data from the same hospital, but perform significantly worse when tested in different hospital systems. We propose an approach based on adversarial optimization, which allows us to learn more robust models that do not depend on confounders.
  arXiv  Detail & Related papers  (2020-01-13T03:49:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.