RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging

• URL: http://arxiv.org/abs/2210.08388v1
• Date: Sat, 15 Oct 2022 22:32:20 GMT
• Title: RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging
• Authors: Ajay Jaiswal, Kumar Ashutosh, Justin F Rousseau, Yifan Peng, Zhangyang Wang, and Ying Ding
• Abstract summary: Deep learning models trained on noisy datasets are sensitive to the noise type and lead to less generalization on unseen samples. We propose a Robust Knowledge Distillation (RoS-KD) framework which mimics the notion of learning a topic from multiple sources to ensure deterrence in learning noisy information. RoS-KD learns a smooth, well-informed, and robust student manifold by distilling knowledge from multiple teachers trained on overlapping subsets of training data.
• Score: 67.02500668641831
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI-powered Medical Imaging has recently achieved enormous attention due to its ability to provide fast-paced healthcare diagnoses. However, it usually suffers from a lack of high-quality datasets due to high annotation cost, inter-observer variability, human annotator error, and errors in computer-generated labels. Deep learning models trained on noisy labelled datasets are sensitive to the noise type and lead to less generalization on the unseen samples. To address this challenge, we propose a Robust Stochastic Knowledge Distillation (RoS-KD) framework which mimics the notion of learning a topic from multiple sources to ensure deterrence in learning noisy information. More specifically, RoS-KD learns a smooth, well-informed, and robust student manifold by distilling knowledge from multiple teachers trained on overlapping subsets of training data. Our extensive experiments on popular medical imaging classification tasks (cardiopulmonary disease and lesion classification) using real-world datasets, show the performance benefit of RoS-KD, its ability to distill knowledge from many popular large networks (ResNet-50, DenseNet-121, MobileNet-V2) in a comparatively small network, and its robustness to adversarial attacks (PGD, FSGM). More specifically, RoS-KD achieves >2% and >4% improvement on F1-score for lesion classification and cardiopulmonary disease classification tasks, respectively, when the underlying student is ResNet-18 against recent competitive knowledge distillation baseline. Additionally, on cardiopulmonary disease classification task, RoS-KD outperforms most of the SOTA baselines by ~1% gain in AUC score.

Related papers

• Deep Learning-Based Assessment of Cerebral Microbleeds in COVID-19 [0.5382679710017695]
  Cerebral Microbleeds (CMBs) are captured as hypointensities from susceptibility-weighted imaging (SWI) Recent studies on COVID-19 have shown an increase in CMBs of coronavirus cases. Existing deep learning methods are mostly trained on very limited research data. We propose an efficient 3D deep learning framework that is actively trained on multi-domain data.
  arXiv  Detail & Related papers  (2023-01-23T08:46:17Z)
• Machine Learning Performance Analysis to Predict Stroke Based on Imbalanced Medical Dataset [0.0]
  Cerebral stroke, the second most substantial cause of death universally, has been a primary public health concern over the last few years. Medical dataset are frequently unbalanced in their class label, with a tendency to poorly predict minority classes. In this paper, the potential risk factors for stroke are investigated. Four distinctive approaches are applied to improve the classification of the minority class in the imbalanced stroke dataset.
  arXiv  Detail & Related papers  (2022-11-14T17:36:46Z)
• Knowledge distillation with a class-aware loss for endoscopic disease detection [1.1470070927586016]
  In this work, we leverage deep learning to develop a framework to improve the localization of difficult to detect lesions. Our model achieves higher performance in terms of mean average precision (mAP) on both endoscopic disease detection challenge and Kvasir-SEG datasets.
  arXiv  Detail & Related papers  (2022-07-19T19:56:12Z)
• When Accuracy Meets Privacy: Two-Stage Federated Transfer Learning Framework in Classification of Medical Images on Limited Data: A COVID-19 Case Study [77.34726150561087]
  COVID-19 pandemic has spread rapidly and caused a shortage of global medical resources. CNN has been widely utilized and verified in analyzing medical images.
  arXiv  Detail & Related papers  (2022-03-24T02:09:41Z)
• SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images [62.60956024215873]
  Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. This study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin diseases classification.
  arXiv  Detail & Related papers  (2022-03-22T06:54:29Z)
• Machine Learning-Based Classification Algorithms for the Prediction of Coronary Heart Diseases [0.0]
  The study created and tested several machine-learning-based classification models. The results show that logistic regression produced the highest performance score on the original dataset. In conclusion, this study suggests that LR on a well-processed and standardized dataset can predict coronary heart disease with greater accuracy than the other algorithms.
  arXiv  Detail & Related papers  (2021-12-02T18:52:56Z)
• Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation [72.0276067144762]
  We discuss and define the two common types of label noise in medical images. We propose an uncertainty estimation-based framework to handle these two label noise amid the medical image classification task.
  arXiv  Detail & Related papers  (2021-02-28T14:56:45Z)
• A Benchmark for Studying Diabetic Retinopathy: Segmentation, Grading, and Transferability [76.64661091980531]
  People with diabetes are at risk of developing diabetic retinopathy (DR) Computer-aided DR diagnosis is a promising tool for early detection of DR and severity grading. This dataset has 1,842 images with pixel-level DR-related lesion annotations, and 1,000 images with image-level labels graded by six board-certified ophthalmologists.
  arXiv  Detail & Related papers  (2020-08-22T07:48:04Z)
• Deep Multi-Scale Resemblance Network for the Sub-class Differentiation of Adrenal Masses on Computed Tomography Images [16.041873352037594]
  Adrenal masses can be benign or malignant and benign masses have varying prevalence. CNNs are the state-of-the-art in maximizing inter-class differences in large medical imaging training datasets. The application of CNNs, to adrenal masses is challenging due to large intra-class variations, large inter-class similarities and imbalanced training data. We developed a deep multi-scale resemblance network (DMRN) to overcome these limitations and leveraged paired CNNs to evaluate the intra-class similarities.
  arXiv  Detail & Related papers  (2020-07-29T06:24:53Z)
• Multi-label Thoracic Disease Image Classification with Cross-Attention Networks [65.37531731899837]
  We propose a novel scheme of Cross-Attention Networks (CAN) for automated thoracic disease classification from chest x-ray images. We also design a new loss function that beyond cross-entropy loss to help cross-attention process and is able to overcome the imbalance between classes and easy-dominated samples within each class.
  arXiv  Detail & Related papers  (2020-07-21T14:37:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.