Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification

• URL: http://arxiv.org/abs/2601.12826v1
• Date: Mon, 19 Jan 2026 08:35:59 GMT
• Title: Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification
• Authors: Teerapong Panboonyuen,
• Abstract summary: This study investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification.<n>We introduce a quantitative evaluation framework that combines localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability.<n>Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to "trust" a model's explanation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (Grad-CAM), have become indispensable for visualizing the reasoning process of deep neural networks in medical image analysis. Despite their popularity, the faithfulness and reliability of these heatmap-based explanations remain under scrutiny. This study critically investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification. Using the publicly available IQ-OTH/NCCD dataset, we evaluate five representative architectures: ResNet-50, ResNet-101, DenseNet-161, EfficientNet-B0, and ViT-Base-Patch16-224, to explore model-dependent variations in Grad-CAM interpretability. We introduce a quantitative evaluation framework that combines localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability across architectures. Experimental findings reveal that while Grad-CAM effectively highlights salient tumor regions in most convolutional networks, its interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior. Furthermore, cross-model comparisons indicate substantial variability in saliency localization, implying that Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks. This work exposes critical limitations of current saliency-based XAI approaches in medical imaging and emphasizes the need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to "trust" a model's explanation.

Related papers

• UbiQVision: Quantifying Uncertainty in XAI for Image Recognition [39.47298454012977]
  SHAP explanations can be unstable and unreliable in the presence of epistemic and aleatoric uncertainty.<n>This study uses Dirichlet posterior sampling and Dempster-Shafer theory to quantify the uncertainty that arises from these unstable explanations in medical imaging applications.
  arXiv  Detail & Related papers  (2025-12-23T11:57:34Z)
• Interpretable Clinical Classification with Kolgomorov-Arnold Networks [70.72819760172744]
  Kolmogorov-Arnold Networks (KANs) offer intrinsic interpretability through transparent, symbolic representations.<n>KANs support built-in patient-level insights, intuitive visualizations, and nearest-patient retrieval.<n>These results position KANs as a promising step toward trustworthy AI that clinicians can understand, audit, and act upon.
  arXiv  Detail & Related papers  (2025-09-20T17:21:58Z)
• Enhanced SegNet with Integrated Grad-CAM for Interpretable Retinal Layer Segmentation in OCT Images [0.0]
  This study proposes an improved SegNet-based deep learning framework for automated and interpretable retinal layer segmentation.<n> Architectural innovations, including modified pooling strategies, enhance feature extraction from noisy OCT images.<n>Grad-CAM visualizations highlighted anatomically relevant regions, aligning segmentation with clinical biomarkers.
  arXiv  Detail & Related papers  (2025-09-09T14:31:51Z)
• A Foundation Model for Chest X-ray Interpretation with Grounded Reasoning via Online Reinforcement Learning [41.27625400846057]
  DeepMedix-R1 is a holistic medical FM for chest X-ray (CXR) interpretation.<n>It generates both an answer and reasoning steps tied to the image's local regions for each query.
  arXiv  Detail & Related papers  (2025-09-04T06:00:04Z)
• Benchmarking Class Activation Map Methods for Explainable Brain Hemorrhage Classification on Hemorica Dataset [0.0]
  This study investigates brain hemorrhage diagnosis with a focus on explainability through Class Activation Mapping (CAM) techniques.<n>A pipeline was developed to extract pixellevel segmentation and detection annotations from classification models using nine state-of-the-art CAM algorithms.
  arXiv  Detail & Related papers  (2025-08-25T06:16:32Z)
• A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans [0.0]
  This study presents a systematic evaluation of various deep learning architectures combined with explainable AI (xAI) techniques for automated region of interest detection in knee MRI scans.<n>We investigate both supervised and self-supervised approaches, including ResNet50, InceptionV3, Vision Transformers (ViT), and multiple U-Net variants augmented with multi-layer perceptron (MLP) classifiers.<n>Our results demonstrate that ResNet50 consistently excels in classification and ROI identification, outperforming transformer-based models under the constraints of the MRNet dataset.
  arXiv  Detail & Related papers  (2025-08-19T17:42:45Z)
• Analyzing the Effect of $k$-Space Features in MRI Classification Models [0.0]
  We have developed an explainable AI methodology tailored for medical imaging. We employ a Convolutional Neural Network (CNN) that analyzes MRI scans across both image and frequency domains. This approach not only enhances early training efficiency but also deepens our understanding of how additional features impact the model predictions.
  arXiv  Detail & Related papers  (2024-09-20T15:43:26Z)
• Classification of lung cancer subtypes on CT images with synthetic pathological priors [41.75054301525535]
  Cross-scale associations exist in the image patterns between the same case's CT images and its pathological images. We propose self-generating hybrid feature network (SGHF-Net) for accurately classifying lung cancer subtypes on CT images.
  arXiv  Detail & Related papers  (2023-08-09T02:04:05Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• InDuDoNet+: A Model-Driven Interpretable Dual Domain Network for Metal Artifact Reduction in CT Images [53.4351366246531]
  We construct a novel interpretable dual domain network, termed InDuDoNet+, into which CT imaging process is finely embedded. We analyze the CT values among different tissues, and merge the prior observations into a prior network for our InDuDoNet+, which significantly improve its generalization performance.
  arXiv  Detail & Related papers  (2021-12-23T15:52:37Z)
• Learning Interpretable Microscopic Features of Tumor by Multi-task Adversarial CNNs To Improve Generalization [1.7371375427784381]
  Existing CNN models act as black boxes, not ensuring to the physicians that important diagnostic features are used by the model. Here we show that our architecture, by learning end-to-end an uncertainty-based weighting combination of multi-task and adversarial losses, is encouraged to focus on pathology features. Our results on breast lymph node tissue show significantly improved generalization in the detection of tumorous tissue, with best average AUC 0.89 (0.01) against the baseline AUC 0.86 (0.005)
  arXiv  Detail & Related papers  (2020-08-04T12:10:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.