A Knowledge Distillation-Based Approach to Enhance Transparency of Classifier Models

• URL: http://arxiv.org/abs/2502.15959v1
• Date: Fri, 21 Feb 2025 21:43:21 GMT
• Title: A Knowledge Distillation-Based Approach to Enhance Transparency of Classifier Models
• Authors: Yuchen Jiang, Xinyuan Zhao, Yihang Wu, Ahmad Chaddad,
• Abstract summary: In medical image analysis, a high degree of transparency and model interpretability can help clinicians better understand and trust the decision-making process of AI models.<n>We propose a Knowledge Distillation (KD)-based approach that aims to enhance the transparency of the AI model in medical image analysis.
• Score: 5.8996922379678
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the rapid development of artificial intelligence (AI), especially in the medical field, the need for its explainability has grown. In medical image analysis, a high degree of transparency and model interpretability can help clinicians better understand and trust the decision-making process of AI models. In this study, we propose a Knowledge Distillation (KD)-based approach that aims to enhance the transparency of the AI model in medical image analysis. The initial step is to use traditional CNN to obtain a teacher model and then use KD to simplify the CNN architecture, retain most of the features of the data set, and reduce the number of network layers. It also uses the feature map of the student model to perform hierarchical analysis to identify key features and decision-making processes. This leads to intuitive visual explanations. We selected three public medical data sets (brain tumor, eye disease, and Alzheimer's disease) to test our method. It shows that even when the number of layers is reduced, our model provides a remarkable result in the test set and reduces the time required for the interpretability analysis.

Related papers

• Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks [9.93411316886105]
  Self-eXplainable AI (S-XAI) incorporates explainability directly into the training process of deep learning models. This survey presents a comprehensive review across various image modalities and clinical applications.
  arXiv  Detail & Related papers  (2024-10-03T09:29:28Z)
• Analyzing the Effect of $k$-Space Features in MRI Classification Models [0.0]
  We have developed an explainable AI methodology tailored for medical imaging. We employ a Convolutional Neural Network (CNN) that analyzes MRI scans across both image and frequency domains. This approach not only enhances early training efficiency but also deepens our understanding of how additional features impact the model predictions.
  arXiv  Detail & Related papers  (2024-09-20T15:43:26Z)
• Decoding Decision Reasoning: A Counterfactual-Powered Model for Knowledge Discovery [6.1521675665532545]
  In medical imaging, discerning the rationale behind an AI model's predictions is crucial for evaluating its reliability. We propose an explainable model that is equipped with both decision reasoning and feature identification capabilities. By implementing our method, we can efficiently identify and visualise class-specific features leveraged by the data-driven model.
  arXiv  Detail & Related papers  (2024-05-23T19:00:38Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Pixel-Level Explanation of Multiple Instance Learning Models in Biomedical Single Cell Images [52.527733226555206]
  We investigate the use of four attribution methods to explain a multiple instance learning models. We study two datasets of acute myeloid leukemia with over 100 000 single cell images. We compare attribution maps with the annotations of a medical expert to see how the model's decision-making differs from the human standard.
  arXiv  Detail & Related papers  (2023-03-15T14:00:11Z)
• Contrastive Brain Network Learning via Hierarchical Signed Graph Pooling Model [64.29487107585665]
  Graph representation learning techniques on brain functional networks can facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. Here, we propose an interpretable hierarchical signed graph representation learning model to extract graph-level representations from brain functional networks. In order to further improve the model performance, we also propose a new strategy to augment functional brain network data for contrastive learning.
  arXiv  Detail & Related papers  (2022-07-14T20:03:52Z)
• A Deep Reinforcement Learning Framework for Rapid Diagnosis of Whole Slide Pathological Images [4.501311544043762]
  We propose a weakly supervised deep reinforcement learning framework, which can greatly reduce the time required for network inference. We use neural network to construct the search model and decision model of reinforcement learning agent respectively. Experimental results show that our proposed method can achieve fast inference and accurate prediction of whole slide images without any pixel-level annotations.
  arXiv  Detail & Related papers  (2022-05-05T14:20:29Z)
• SSD-KD: A Self-supervised Diverse Knowledge Distillation Method for Lightweight Skin Lesion Classification Using Dermoscopic Images [62.60956024215873]
  Skin cancer is one of the most common types of malignancy, affecting a large population and causing a heavy economic burden worldwide. Most studies in skin cancer detection keep pursuing high prediction accuracies without considering the limitation of computing resources on portable devices. This study specifically proposes a novel method, termed SSD-KD, that unifies diverse knowledge into a generic KD framework for skin diseases classification.
  arXiv  Detail & Related papers  (2022-03-22T06:54:29Z)
• Demystifying Deep Learning Models for Retinal OCT Disease Classification using Explainable AI [0.6117371161379209]
  The adoption of various deep learning techniques is quite common as well as effective, and its statement is equally true when it comes to implementing it into the retina Optical Coherence Tomography sector. These techniques have the black box characteristics that prevent the medical professionals to completely trust the results generated from them. This paper proposes a self-developed CNN model which is comparatively smaller and simpler along with the use of Lime that introduces Explainable AI to the study.
  arXiv  Detail & Related papers  (2021-11-06T13:54:07Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.