Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism

• URL: http://arxiv.org/abs/2209.11189v1
• Date: Thu, 22 Sep 2022 17:33:18 GMT
• Title: Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism
• Authors: Ioanna Gkartzonika, Nikolaos Gkalelis, Vasileios Mezaris
• Abstract summary: Two new learning-based AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage.
• Score: 8.395400675921515
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper two new learning-based eXplainable AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. During training, CAMs are applied to the feature maps (L-CAM-Fm) or the input image (L-CAM-Img) forcing the attention mechanism to learn the image regions explaining the DCNN's outcome. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage. Moreover, based on the derived explanations a comprehensive qualitative analysis is performed providing valuable insight for understanding the reasons behind classification errors, including possible dataset biases affecting the trained classifier.

Related papers

• KPCA-CAM: Visual Explainability of Deep Computer Vision Models using Kernel PCA [1.5550533143704957]
  This research introduces KPCA-CAM, a technique designed to enhance the interpretability of Convolutional Neural Networks (CNNs) KPCA-CAM leverages Principal Component Analysis (PCA) with the kernel trick to capture nonlinear relationships within CNN activations more effectively. Empirical evaluations on the ILSVRC dataset across different CNN models demonstrate that KPCA-CAM produces more precise activation maps.
  arXiv  Detail & Related papers  (2024-09-30T22:36:37Z)
• DCNN: Dual Cross-current Neural Networks Realized Using An Interactive Deep Learning Discriminator for Fine-grained Objects [48.65846477275723]
  This study proposes novel dual-current neural networks (DCNN) to improve the accuracy of fine-grained image classification. The main novel design features for constructing a weakly supervised learning backbone model DCNN include (a) extracting heterogeneous data, (b) keeping the feature map resolution unchanged, (c) expanding the receptive field, and (d) fusing global representations and local features.
  arXiv  Detail & Related papers  (2024-05-07T07:51:28Z)
• BroadCAM: Outcome-agnostic Class Activation Mapping for Small-scale Weakly Supervised Applications [69.22739434619531]
  We propose an outcome-agnostic CAM approach, called BroadCAM, for small-scale weakly supervised applications. By evaluating BroadCAM on VOC2012 and BCSS-WSSS for WSSS and OpenImages30k for WSOL, BroadCAM demonstrates superior performance.
  arXiv  Detail & Related papers  (2023-09-07T06:45:43Z)
• An Explainable Model-Agnostic Algorithm for CNN-based Biometrics Verification [55.28171619580959]
  This paper describes an adaptation of the Local Interpretable Model-Agnostic Explanations (LIME) AI method to operate under a biometric verification setting.
  arXiv  Detail & Related papers  (2023-07-25T11:51:14Z)
• Feature Activation Map: Visual Explanation of Deep Learning Models for Image Classification [17.373054348176932]
  In this work, a post-hoc interpretation tool named feature activation map (FAM) is proposed. FAM can interpret deep learning models without FC layers as a classifier. Experiments conducted on ten deep learning models for few-shot image classification, contrastive learning image classification and image retrieval tasks demonstrate the effectiveness of the proposed FAM algorithm.
  arXiv  Detail & Related papers  (2023-07-11T05:33:46Z)
• VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network [10.365366151667017]
  Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks. For standard convolutional neural networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN's decision and image region by generating a heatmap. In this paper, we proposed a novel visualization method particularly applicable to GCN, Vertex Semantic Class Activation Mapping (VS-CAM)
  arXiv  Detail & Related papers  (2022-09-15T09:45:59Z)
• Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value [86.69600830581912]
  We develop a novel visual explanation method called Shap-CAM based on class activation mapping. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process.
  arXiv  Detail & Related papers  (2022-08-07T00:59:23Z)
• Towards Learning Spatially Discriminative Feature Representations [26.554140976236052]
  We propose a novel loss function, termed as CAM-loss, to constrain the embedded feature maps with the class activation maps (CAMs) CAM-loss drives the backbone to express the features of target category and suppress the features of non-target categories or background. Experimental results show that CAM-loss is applicable to a variety of network structures and can be combined with mainstream regularization methods to improve the performance of image classification.
  arXiv  Detail & Related papers  (2021-09-03T08:04:17Z)
• Learning CNN filters from user-drawn image markers for coconut-tree image classification [78.42152902652215]
  We present a method that needs a minimal set of user-selected images to train the CNN's feature extractor. The method learns the filters of each convolutional layer from user-drawn markers in image regions that discriminate classes. It does not rely on optimization based on backpropagation, and we demonstrate its advantages on the binary classification of coconut-tree aerial images.
  arXiv  Detail & Related papers  (2020-08-08T15:50:23Z)
• Eigen-CAM: Class Activation Map using Principal Components [1.2691047660244335]
  This paper builds on previous ideas to cope with the increasing demand for interpretable, robust, and transparent models. The proposed Eigen-CAM computes and visualizes the principle components of the learned features/representations from the convolutional layers.
  arXiv  Detail & Related papers  (2020-08-01T17:14:13Z)
• Ventral-Dorsal Neural Networks: Object Detection via Selective Attention [51.79577908317031]
  We propose a new framework called Ventral-Dorsal Networks (VDNets) Inspired by the structure of the human visual system, we propose the integration of a "Ventral Network" and a "Dorsal Network" Our experimental results reveal that the proposed method outperforms state-of-the-art object detection approaches.
  arXiv  Detail & Related papers  (2020-05-15T23:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.