On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices

• URL: http://arxiv.org/abs/2212.08130v1
• Date: Thu, 15 Dec 2022 20:35:48 GMT
• Title: On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices
• Authors: Salah Ghamizi, Maxime Cordy, Michail Papadakis, and Yves Le Traon
• Abstract summary: We show that robustness of chest x-ray classification is much harder to evaluate than natural images. We argue that previous studies did not take into account the peculiarity of medical diagnosis. Our evaluation on 3 datasets, 7 models, and 18 diseases is the largest evaluation of robustness of chest x-ray classification models.
• Score: 9.142684157074498
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Vulnerability to adversarial attacks is a well-known weakness of Deep Neural Networks. While most of the studies focus on natural images with standardized benchmarks like ImageNet and CIFAR, little research has considered real world applications, in particular in the medical domain. Our research shows that, contrary to previous claims, robustness of chest x-ray classification is much harder to evaluate and leads to very different assessments based on the dataset, the architecture and robustness metric. We argue that previous studies did not take into account the peculiarity of medical diagnosis, like the co-occurrence of diseases, the disagreement of labellers (domain experts), the threat model of the attacks and the risk implications for each successful attack. In this paper, we discuss the methodological foundations, review the pitfalls and best practices, and suggest new methodological considerations for evaluating the robustness of chest xray classification models. Our evaluation on 3 datasets, 7 models, and 18 diseases is the largest evaluation of robustness of chest x-ray classification models.

Related papers

• A Survey of Neural Network Robustness Assessment in Image Recognition [4.581878177334397]
  In recent years, there has been significant attention given to the robustness assessment of neural networks. Deep learning's robustness problem is particularly significant, highlighted by the discovery of adversarial attacks on image classification models. In this survey, we present a detailed examination of both adversarial robustness (AR) and corruption robustness (CR) in neural network assessment.
  arXiv  Detail & Related papers  (2024-04-12T07:19:16Z)
• Robust and Interpretable COVID-19 Diagnosis on Chest X-ray Images using Adversarial Training [0.8287206589886881]
  We train 21 convolutional neural network (CNN) models on a diverse set of 33,000+ chest X-ray (CXR) images to classify between healthy, COVID-19, and non-COVID-19 pneumonia CXRs. Our resulting models achieved a 3-way classification accuracy, recall, and precision of up to 97.03%, 97.97%, and 99.95%, respectively.
  arXiv  Detail & Related papers  (2023-11-23T23:40:01Z)
• Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders [50.689585476660554]
  We propose a new fine-tuning strategy that includes positive-pair loss relaxation and random sentence sampling. Our approach consistently improves overall zero-shot pathology classification across four chest X-ray datasets and three pre-trained models.
  arXiv  Detail & Related papers  (2022-12-14T06:04:18Z)
• Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study [75.05049024176584]
  We present a benchmark study of the long-tailed learning problem in the specific domain of thorax diseases on chest X-rays. We focus on learning from naturally distributed chest X-ray data, optimizing classification accuracy over not only the common "head" classes, but also the rare yet critical "tail" classes. The benchmark consists of two chest X-ray datasets for 19- and 20-way thorax disease classification, containing classes with as many as 53,000 and as few as 7 labeled training images.
  arXiv  Detail & Related papers  (2022-08-29T04:34:15Z)
• POTHER: Patch-Voted Deep Learning-based Chest X-ray Bias Analysis for COVID-19 Detection [10.516962652888989]
  Many studies reported detecting COVID-19 in chest X-rays accurately using deep learning. We demonstrate that model decisions may rely on confounding factors rather than medical pathology. We propose a novel method to minimise their negative impact.
  arXiv  Detail & Related papers  (2022-01-23T20:35:45Z)
• Knee Osteoarthritis Severity Prediction using an Attentive Multi-Scale Deep Convolutional Neural Network [8.950918531231158]
  This paper presents a deep learning-based framework, namely OsteoHRNet, that automatically assesses the Knee Osteoarthritis severity in terms of Kellgren and Lawrence grade classification from X-rays. Our proposed model has achieved the best multiclass accuracy of 71.74% and MAE of 0.311 on the baseline cohort of the OAI dataset.
  arXiv  Detail & Related papers  (2021-06-27T17:29:46Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack [38.480886577088384]
  Recent studies have shown deep diagnostic models may not be robust in the inference process. Adversarial example is a well-designed perturbation that is not easily perceived by humans. We have designed two new defense methods to handle adversarial examples in deep diagnostic models.
  arXiv  Detail & Related papers  (2021-03-05T02:24:47Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)
• A Thorough Comparison Study on Adversarial Attacks and Defenses for Common Thorax Disease Classification in Chest X-rays [63.675522663422896]
  We review various adversarial attack and defense methods on chest X-rays. We find that the attack and defense methods have poor performance with excessive iterations and large perturbations. We propose a new defense method that is robust to different degrees of perturbations.
  arXiv  Detail & Related papers  (2020-03-31T06:21:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.