Feature Activation Map: Visual Explanation of Deep Learning Models for Image Classification

• URL: http://arxiv.org/abs/2307.05017v1
• Date: Tue, 11 Jul 2023 05:33:46 GMT
• Title: Feature Activation Map: Visual Explanation of Deep Learning Models for Image Classification
• Authors: Yi Liao, Yongsheng Gao, Weichuan Zhang
• Abstract summary: In this work, a post-hoc interpretation tool named feature activation map (FAM) is proposed. FAM can interpret deep learning models without FC layers as a classifier. Experiments conducted on ten deep learning models for few-shot image classification, contrastive learning image classification and image retrieval tasks demonstrate the effectiveness of the proposed FAM algorithm.
• Score: 17.373054348176932
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Decisions made by convolutional neural networks(CNN) can be understood and explained by visualizing discriminative regions on images. To this end, Class Activation Map (CAM) based methods were proposed as powerful interpretation tools, making the prediction of deep learning models more explainable, transparent, and trustworthy. However, all the CAM-based methods (e.g., CAM, Grad-CAM, and Relevance-CAM) can only be used for interpreting CNN models with fully-connected (FC) layers as a classifier. It is worth noting that many deep learning models classify images without FC layers, e.g., few-shot learning image classification, contrastive learning image classification, and image retrieval tasks. In this work, a post-hoc interpretation tool named feature activation map (FAM) is proposed, which can interpret deep learning models without FC layers as a classifier. In the proposed FAM algorithm, the channel-wise contribution weights are derived from the similarity scores between two image embeddings. The activation maps are linearly combined with the corresponding normalized contribution weights, forming the explanation map for visualization. The quantitative and qualitative experiments conducted on ten deep learning models for few-shot image classification, contrastive learning image classification and image retrieval tasks demonstrate the effectiveness of the proposed FAM algorithm.

Related papers

• Linking in Style: Understanding learned features in deep learning models [0.0]
  Convolutional neural networks (CNNs) learn abstract features to perform object classification. We propose an automatic method to visualize and systematically analyze learned features in CNNs.
  arXiv  Detail & Related papers  (2024-09-25T12:28:48Z)
• LICO: Explainable Models with Language-Image Consistency [39.869639626266554]
  This paper develops a Language-Image COnsistency model for explainable image classification, termed LICO. We first establish a coarse global manifold structure alignment by minimizing the distance between the distributions of image and language features. We then achieve fine-grained saliency maps by applying optimal transport (OT) theory to assign local feature maps with class-specific prompts.
  arXiv  Detail & Related papers  (2023-10-15T12:44:33Z)
• An Explainable Model-Agnostic Algorithm for CNN-based Biometrics Verification [55.28171619580959]
  This paper describes an adaptation of the Local Interpretable Model-Agnostic Explanations (LIME) AI method to operate under a biometric verification setting.
  arXiv  Detail & Related papers  (2023-07-25T11:51:14Z)
• Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism [8.395400675921515]
  Two new learning-based AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage.
  arXiv  Detail & Related papers  (2022-09-22T17:33:18Z)
• Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training of Image Segmentation Models [54.49581189337848]
  We propose a method to enable the end-to-end pre-training for image segmentation models based on classification datasets. The proposed method leverages a weighted segmentation learning procedure to pre-train the segmentation network en masse. Experiment results show that, with ImageNet accompanied by PSSL as the source dataset, the proposed end-to-end pre-training strategy successfully boosts the performance of various segmentation models.
  arXiv  Detail & Related papers  (2022-07-04T13:02:32Z)
• Adaptive Convolutional Dictionary Network for CT Metal Artifact Reduction [62.691996239590125]
  We propose an adaptive convolutional dictionary network (ACDNet) for metal artifact reduction. Our ACDNet can automatically learn the prior for artifact-free CT images via training data and adaptively adjust the representation kernels for each input CT image. Our method inherits the clear interpretability of model-based methods and maintains the powerful representation ability of learning-based methods.
  arXiv  Detail & Related papers  (2022-05-16T06:49:36Z)
• Convolutional Neural Networks from Image Markers [62.997667081978825]
  Feature Learning from Image Markers (FLIM) was recently proposed to estimate convolutional filters, with no backpropagation, from strokes drawn by a user on very few images. This paper extends FLIM for fully connected layers and demonstrates it on different image classification problems. The results show that FLIM-based convolutional neural networks can outperform the same architecture trained from scratch by backpropagation.
  arXiv  Detail & Related papers  (2020-12-15T22:58:23Z)
• PK-GCN: Prior Knowledge Assisted Image Classification using Graph Convolution Networks [3.4129083593356433]
  Similarity between classes can influence the performance of classification. We propose a method that incorporates class similarity knowledge into convolutional neural networks models. Experimental results show that our model can improve classification accuracy, especially when the amount of available data is small.
  arXiv  Detail & Related papers  (2020-09-24T18:31:35Z)
• Region Comparison Network for Interpretable Few-shot Image Classification [97.97902360117368]
  Few-shot image classification has been proposed to effectively use only a limited number of labeled examples to train models for new classes. We propose a metric learning based method named Region Comparison Network (RCN), which is able to reveal how few-shot learning works. We also present a new way to generalize the interpretability from the level of tasks to categories.
  arXiv  Detail & Related papers  (2020-09-08T07:29:05Z)
• Learning CNN filters from user-drawn image markers for coconut-tree image classification [78.42152902652215]
  We present a method that needs a minimal set of user-selected images to train the CNN's feature extractor. The method learns the filters of each convolutional layer from user-drawn markers in image regions that discriminate classes. It does not rely on optimization based on backpropagation, and we demonstrate its advantages on the binary classification of coconut-tree aerial images.
  arXiv  Detail & Related papers  (2020-08-08T15:50:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.