VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network

• URL: http://arxiv.org/abs/2209.09104v1
• Date: Thu, 15 Sep 2022 09:45:59 GMT
• Title: VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision Graph Neural Network
• Authors: Zhenpeng Feng, Xiyang Cui, Hongbing Ji, Mingzhe Zhu, Ljubisa Stankovic
• Abstract summary: Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks. For standard convolutional neural networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN's decision and image region by generating a heatmap. In this paper, we proposed a novel visualization method particularly applicable to GCN, Vertex Semantic Class Activation Mapping (VS-CAM)
• Score: 10.365366151667017
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks, however, there lacks a clear interpretation of GCN's inner mechanism. For standard convolutional neural networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN's decision and image region by generating a heatmap. Nonetheless, such heatmap usually exhibits semantic-chaos when these CAMs are applied to GCN directly. In this paper, we proposed a novel visualization method particularly applicable to GCN, Vertex Semantic Class Activation Mapping (VS-CAM). VS-CAM includes two independent pipelines to produce a set of semantic-probe maps and a semantic-base map, respectively. Semantic-probe maps are used to detect the semantic information from semantic-base map to aggregate a semantic-aware heatmap. Qualitative results show that VS-CAM can obtain heatmaps where the highlighted regions match the objects much more precisely than CNN-based CAM. The quantitative evaluation further demonstrates the superiority of VS-CAM.

Related papers

• Leveraging CAM Algorithms for Explaining Medical Semantic Segmentation [4.818865062632567]
  Convolutional neural networks (CNNs) achieve prevailing results in segmentation tasks nowadays. One way of interpreting a CNN is the use of class activation maps (CAMs) that represent heatmaps. We propose a transfer between existing classification- and segmentation-based methods for more detailed, explainable, and consistent results.
  arXiv  Detail & Related papers  (2024-09-30T13:43:00Z)
• Hierarchical Graph Interaction Transformer with Dynamic Token Clustering for Camouflaged Object Detection [57.883265488038134]
  We propose a hierarchical graph interaction network termed HGINet for camouflaged object detection. The network is capable of discovering imperceptible objects via effective graph interaction among the hierarchical tokenized features. Our experiments demonstrate the superior performance of HGINet compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-08-27T12:53:25Z)
• Domain-adaptive Message Passing Graph Neural Network [67.35534058138387]
  Cross-network node classification (CNNC) aims to classify nodes in a label-deficient target network by transferring the knowledge from a source network with abundant labels. We propose a domain-adaptive message passing graph neural network (DM-GNN), which integrates graph neural network (GNN) with conditional adversarial domain adaptation.
  arXiv  Detail & Related papers  (2023-08-31T05:26:08Z)
• Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in Image Classification [12.971559051829658]
  Cluster-CAM is an effective and efficient gradient-free CNN interpretation algorithm. We propose an artful strategy to forge a cognition-base map and cognition-scissors from clustered feature maps.
  arXiv  Detail & Related papers  (2023-02-03T10:38:20Z)
• Opti-CAM: Optimizing saliency maps for interpretability [10.122899813335694]
  We introduce Opti-CAM, combining ideas from CAM-based and masking-based approaches. Our saliency map is a linear combination of feature maps, where weights are optimized per image. On several datasets, Opti-CAM largely outperforms other CAM-based approaches according to the most relevant classification metrics.
  arXiv  Detail & Related papers  (2023-01-17T16:44:48Z)
• Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism [8.395400675921515]
  Two new learning-based AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed. Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps. Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage.
  arXiv  Detail & Related papers  (2022-09-22T17:33:18Z)
• Shap-CAM: Visual Explanations for Convolutional Neural Networks based on Shapley Value [86.69600830581912]
  We develop a novel visual explanation method called Shap-CAM based on class activation mapping. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process.
  arXiv  Detail & Related papers  (2022-08-07T00:59:23Z)
• Towards Learning Spatially Discriminative Feature Representations [26.554140976236052]
  We propose a novel loss function, termed as CAM-loss, to constrain the embedded feature maps with the class activation maps (CAMs) CAM-loss drives the backbone to express the features of target category and suppress the features of non-target categories or background. Experimental results show that CAM-loss is applicable to a variety of network structures and can be combined with mainstream regularization methods to improve the performance of image classification.
  arXiv  Detail & Related papers  (2021-09-03T08:04:17Z)
• TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization [112.46381729542658]
  Weakly supervised object localization (WSOL) is a challenging problem when given image category labels. We introduce the token semantic coupled attention map (TS-CAM) to take full advantage of the self-attention mechanism in visual transformer for long-range dependency extraction.
  arXiv  Detail & Related papers  (2021-03-27T09:43:16Z)
• Video-based Facial Expression Recognition using Graph Convolutional Networks [57.980827038988735]
  We introduce a Graph Convolutional Network (GCN) layer into a common CNN-RNN based model for video-based facial expression recognition. We evaluate our method on three widely-used datasets, CK+, Oulu-CASIA and MMI, and also one challenging wild dataset AFEW8.0.
  arXiv  Detail & Related papers  (2020-10-26T07:31:51Z)
• Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping [71.91734471596432]
  We propose SEG-GRAD-CAM, a gradient-based method for interpreting semantic segmentation. Our method is an extension of the widely-used Grad-CAM method, applied locally to produce heatmaps showing the relevance of individual pixels for semantic segmentation.
  arXiv  Detail & Related papers  (2020-02-26T12:32:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.