Understanding the impact of class imbalance on the performance of chest x-ray image classifiers

• URL: http://arxiv.org/abs/2112.12843v1
• Date: Thu, 23 Dec 2021 20:57:47 GMT
• Title: Understanding the impact of class imbalance on the performance of chest x-ray image classifiers
• Abstract summary: This work aims to understand the impact of class imbalance on the performance of chest x-ray classifiers. We propose the adoption of two alternative metrics, the precision-recall curve and the Balanced Brier score, which better reflect the performance of the system.
• Score: 13.593336529221613
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work aims to understand the impact of class imbalance on the performance of chest x-ray classifiers, in light of the standard evaluation practices adopted by researchers in terms of discrimination and calibration performance. Firstly, we conducted a literature study to analyze common scientific practices and confirmed that: (1) even when dealing with highly imbalanced datasets, the community tends to use metrics that are dominated by the majority class; and (2) it is still uncommon to include calibration studies for chest x-ray classifiers, albeit its importance in the context of healthcare. Secondly, we perform a systematic experiment on two major chest x-ray datasets to explore the behavior of several performance metrics under different class ratios and show that widely adopted metrics can conceal the performance in the minority class. Finally, we propose the adoption of two alternative metrics, the precision-recall curve and the Balanced Brier score, which better reflect the performance of the system in such scenarios. Our results indicate that current evaluation practices adopted by the research community for chest x-ray classifiers may not reflect the performance of such systems for computer-aided diagnosis in real clinical scenarios, and suggest alternatives to improve this situation.

Related papers

• From Detection to Mitigation: Addressing Bias in Deep Learning Models for Chest X-Ray Diagnosis [3.9236980712240217]
  We present a comprehensive bias detection and mitigation framework targeting sex, age, and race-based disparities when performing diagnostic tasks with chest X-rays.<n>We show that replacing the final layer of CNN with an eXtreme Gradient Boosting classifier improves the fairness of the subgroup while maintaining or improving the overall predictive performance.<n>We further compare this lightweight adapter training method with traditional full-model training bias mitigation techniques, including adversarial training, reweighting, data augmentation, and active learning, and find that our approach offers competitive or superior bias reduction at a fraction of the computational cost.
  arXiv  Detail & Related papers  (2025-10-12T22:20:08Z)
• Fairness Evolution in Continual Learning for Medical Imaging [47.52603262576663]
  We study the behavior of Continual Learning (CL) strategies in medical imaging regarding classification performance. We evaluate the Replay, Learning without Forgetting (LwF), LwF, and Pseudo-Label strategies. LwF and Pseudo-Label exhibit optimal classification performance, but when including fairness metrics in the evaluation, it is clear that Pseudo-Label is less biased.
  arXiv  Detail & Related papers  (2024-04-10T09:48:52Z)
• Comparison of Machine Learning Classification Algorithms and Application to the Framingham Heart Study [0.0]
  The use of machine learning algorithms in healthcare can amplify social injustices and health inequities. This research pertains to some generalizability impediments that occur during the development and the post-deployment of machine learning classification algorithms.
  arXiv  Detail & Related papers  (2024-02-22T22:49:35Z)
• Multi-task Explainable Skin Lesion Classification [54.76511683427566]
  We propose a few-shot-based approach for skin lesions that generalizes well with few labelled data. The proposed approach comprises a fusion of a segmentation network that acts as an attention module and classification network.
  arXiv  Detail & Related papers  (2023-10-11T05:49:47Z)
• A Survey of the Impact of Self-Supervised Pretraining for Diagnostic Tasks with Radiological Images [71.26717896083433]
  Self-supervised pretraining has been observed to be effective at improving feature representations for transfer learning. This review summarizes recent research into its usage in X-ray, computed tomography, magnetic resonance, and ultrasound imaging.
  arXiv  Detail & Related papers  (2023-09-05T19:45:09Z)
• Discrimination of Radiologists Utilizing Eye-Tracking Technology and Machine Learning: A Case Study [0.9142067094647588]
  This study presents a novel discretized feature encoding based on binning fixation data for efficient geometric alignment. The encoded features of the eye-fixation data are employed by machine learning classifiers to discriminate between faculty and trainee radiologists.
  arXiv  Detail & Related papers  (2023-08-04T23:51:47Z)
• Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective [51.70661197256033]
  We propose ARCO, a semi-supervised contrastive learning framework with stratified group theory for medical image segmentation. We first propose building ARCO through the concept of variance-reduced estimation and show that certain variance-reduction techniques are particularly beneficial in pixel/voxel-level segmentation tasks. We experimentally validate our approaches on eight benchmarks, i.e., five 2D/3D medical and three semantic segmentation datasets, with different label settings.
  arXiv  Detail & Related papers  (2023-02-03T13:50:25Z)
• Density-Aware Personalized Training for Risk Prediction in Imbalanced Medical Data [89.79617468457393]
  Training models with imbalance rate (class density discrepancy) may lead to suboptimal prediction. We propose a framework for training models for this imbalance issue. We demonstrate our model's improved performance in real-world medical datasets.
  arXiv  Detail & Related papers  (2022-07-23T00:39:53Z)
• Learning Discriminative Representation via Metric Learning for Imbalanced Medical Image Classification [52.94051907952536]
  We propose embedding metric learning into the first stage of the two-stage framework specially to help the feature extractor learn to extract more discriminative feature representations. Experiments mainly on three medical image datasets show that the proposed approach consistently outperforms existing onestage and two-stage approaches.
  arXiv  Detail & Related papers  (2022-07-14T14:57:01Z)
• Improving Classification Model Performance on Chest X-Rays through Lung Segmentation [63.45024974079371]
  We propose a deep learning approach to enhance abnormal chest x-ray (CXR) identification performance through segmentations. Our approach is designed in a cascaded manner and incorporates two modules: a deep neural network with criss-cross attention modules (XLSor) for localizing lung region in CXR images and a CXR classification model with a backbone of a self-supervised momentum contrast (MoCo) model pre-trained on large-scale CXR data sets.
  arXiv  Detail & Related papers  (2022-02-22T15:24:06Z)
• Concentration Inequalities for Two-Sample Rank Processes with Application to Bipartite Ranking [0.0]
  The ROC curve is the gold standard for measuring the performance of a test/scoring statistic. Most practical performance measures used in scoring/ranking applications can be viewed as summaries of the ROC curve.
  arXiv  Detail & Related papers  (2021-04-07T06:31:06Z)
• Interpretable COVID-19 Chest X-Ray Classification via Orthogonality Constraint [10.190872613479632]
  We investigate the benefit of using Orthogonal Spheres (OS) constraint for classification of COVID-19 cases from chest X-ray images. Previous studies have demonstrated significant benefits in applying such constraints to deep learning models. Our approach achieves an improvement in accuracy of 1.6% and 4.8% for two- and three-class classification.
  arXiv  Detail & Related papers  (2021-02-02T11:35:28Z)
• Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalization [2.1699022621790736]
  We let six radiologists rate the interpretability of saliency maps in datasets of X-rays, computed tomography, and magnetic resonance imaging scans. We found that the accuracy of adversarially trained models was equal to standard models when sufficiently large datasets and dual batch norm training were used.
  arXiv  Detail & Related papers  (2020-11-25T20:41:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.