Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation

• URL: http://arxiv.org/abs/2409.20287v1
• Date: Mon, 30 Sep 2024 13:43:00 GMT
• Title: Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation
• Authors: Tillmann Rheude, Andreas Wirtz, Arjan Kuijper, Stefan Wesarg,
• Abstract summary: Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays. One way of interpreting a CNN is the use of class activation maps (CAMs) that represent heatmaps. We propose a transfer between existing classification- and segmentation-based methods for more detailed, explainable, and consistent results.
• Score: 4.818865062632567
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays and represent the state-of-the-art for image-based analysis. However, the understanding of the accurate decision-making process of a CNN is rather unknown. The research area of explainable artificial intelligence (xAI) primarily revolves around understanding and interpreting this black-box behavior. One way of interpreting a CNN is the use of class activation maps (CAMs) that represent heatmaps to indicate the importance of image areas for the prediction of the CNN. For classification tasks, a variety of CAM algorithms exist. But for segmentation tasks, only one CAM algorithm for the interpretation of the output of a CNN exist. We propose a transfer between existing classification- and segmentation-based methods for more detailed, explainable, and consistent results which show salient pixels in semantic segmentation tasks. The resulting Seg-HiRes-Grad CAM is an extension of the segmentation-based Seg-Grad CAM with the transfer to the classification-based HiRes CAM. Our method improves the previously-mentioned existing segmentation-based method by adjusting it to recently published classification-based methods. Especially for medical image segmentation, this transfer solves existing explainability disadvantages.

Related papers

• An Explainable Model-Agnostic Algorithm for CNN-based Biometrics Verification [55.28171619580959]
  This paper describes an adaptation of the Local Interpretable Model-Agnostic Explanations (LIME) AI method to operate under a biometric verification setting.
  arXiv  Detail & Related papers  (2023-07-25T11:51:14Z)
• Self-Supervised Correction Learning for Semi-Supervised Biomedical Image Segmentation [84.58210297703714]
  We propose a self-supervised correction learning paradigm for semi-supervised biomedical image segmentation. We design a dual-task network, including a shared encoder and two independent decoders for segmentation and lesion region inpainting. Experiments on three medical image segmentation datasets for different tasks demonstrate the outstanding performance of our method.
  arXiv  Detail & Related papers  (2023-01-12T08:19:46Z)
• Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism [8.395400675921515]
  Two new learning-based AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage.
  arXiv  Detail & Related papers  (2022-09-22T17:33:18Z)
• Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training of Image Segmentation Models [54.49581189337848]
  We propose a method to enable the end-to-end pre-training for image segmentation models based on classification datasets. The proposed method leverages a weighted segmentation learning procedure to pre-train the segmentation network en masse. Experiment results show that, with ImageNet accompanied by PSSL as the source dataset, the proposed end-to-end pre-training strategy successfully boosts the performance of various segmentation models.
  arXiv  Detail & Related papers  (2022-07-04T13:02:32Z)
• A Sneak Attack on Segmentation of Medical Images Using Deep Neural Network Classifiers [0.0]
  We use trained Convolutional Neural Network (CNN) classifiers to approach the segmentation problem. Heatmaps can be visualized and formed using Gradient-weighted Class Activation Mapping (Grad-CAM) Results from our experiments show that heatmaps can locate and segment partial tumor areas.
  arXiv  Detail & Related papers  (2022-01-08T05:57:26Z)
• Weakly-supervised fire segmentation by visualizing intermediate CNN layers [82.75113406937194]
  Fire localization in images and videos is an important step for an autonomous system to combat fire incidents. We consider weakly supervised segmentation of fire in images, in which only image labels are used to train the network. We show that in the case of fire segmentation, which is a binary segmentation problem, the mean value of features in a mid-layer of classification CNN can perform better than conventional Class Activation Mapping (CAM) method.
  arXiv  Detail & Related papers  (2021-11-16T11:56:28Z)
• Towards Learning Spatially Discriminative Feature Representations [26.554140976236052]
  We propose a novel loss function, termed as CAM-loss, to constrain the embedded feature maps with the class activation maps (CAMs) CAM-loss drives the backbone to express the features of target category and suppress the features of non-target categories or background. Experimental results show that CAM-loss is applicable to a variety of network structures and can be combined with mainstream regularization methods to improve the performance of image classification.
  arXiv  Detail & Related papers  (2021-09-03T08:04:17Z)
• The Mind's Eye: Visualizing Class-Agnostic Features of CNNs [92.39082696657874]
  We propose an approach to visually interpret CNN features given a set of images by creating corresponding images that depict the most informative features of a specific layer. Our method uses a dual-objective activation and distance loss, without requiring a generator network nor modifications to the original model.
  arXiv  Detail & Related papers  (2021-01-29T07:46:39Z)
• Region Growing with Convolutional Neural Networks for Biomedical Image Segmentation [1.5469452301122177]
  We present a methodology that uses convolutional neural networks (CNNs) for segmentation by iteratively growing predicted mask regions in each coordinate direction. We use a threshold on the CNN probability scores to determine whether pixels are added to the region and the iteration continues until no new pixels are added to the region. Our method is able to achieve high segmentation accuracy and preserve biologically realistic morphological features while leveraging small amounts of training data and maintaining computational efficiency.
  arXiv  Detail & Related papers  (2020-09-23T17:53:00Z)
• Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping [71.91734471596432]
  We propose SEG-GRAD-CAM, a gradient-based method for interpreting semantic segmentation. Our method is an extension of the widely-used Grad-CAM method, applied locally to produce heatmaps showing the relevance of individual pixels for semantic segmentation.
  arXiv  Detail & Related papers  (2020-02-26T12:32:40Z)
• Edge-Gated CNNs for Volumetric Semantic Segmentation of Medical Images [11.098969286372778]
  In medical imaging, expert manual segmentation often relies on organ boundaries. We propose a plug-and-play module, dubbed Edge-Gated CNNs (EG-CNNs), that can be used with existing encoder-decoder architectures. We evaluate the effectiveness of the EG-CNN with various mainstream CNNs on two publicly available datasets.
  arXiv  Detail & Related papers  (2020-02-11T05:08:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.