VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision
Graph Neural Network
- URL: http://arxiv.org/abs/2209.09104v1
- Date: Thu, 15 Sep 2022 09:45:59 GMT
- Title: VS-CAM: Vertex Semantic Class Activation Mapping to Interpret Vision
Graph Neural Network
- Authors: Zhenpeng Feng, Xiyang Cui, Hongbing Ji, Mingzhe Zhu, Ljubisa Stankovic
- Abstract summary: Graph convolutional neural network (GCN) has drawn increasing attention and attained good performance in various computer vision tasks.
For standard convolutional neural networks (CNNs), class activation mapping (CAM) methods are commonly used to visualize the connection between CNN's decision and image region by generating a heatmap.
In this paper, we proposed a novel visualization method particularly applicable to GCN, Vertex Semantic Class Activation Mapping (VS-CAM)
- Score: 10.365366151667017
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Graph convolutional neural network (GCN) has drawn increasing attention and
attained good performance in various computer vision tasks, however, there
lacks a clear interpretation of GCN's inner mechanism. For standard
convolutional neural networks (CNNs), class activation mapping (CAM) methods
are commonly used to visualize the connection between CNN's decision and image
region by generating a heatmap. Nonetheless, such heatmap usually exhibits
semantic-chaos when these CAMs are applied to GCN directly. In this paper, we
proposed a novel visualization method particularly applicable to GCN, Vertex
Semantic Class Activation Mapping (VS-CAM). VS-CAM includes two independent
pipelines to produce a set of semantic-probe maps and a semantic-base map,
respectively. Semantic-probe maps are used to detect the semantic information
from semantic-base map to aggregate a semantic-aware heatmap. Qualitative
results show that VS-CAM can obtain heatmaps where the highlighted regions
match the objects much more precisely than CNN-based CAM. The quantitative
evaluation further demonstrates the superiority of VS-CAM.
Related papers
- Domain-adaptive Message Passing Graph Neural Network [67.35534058138387]
Cross-network node classification (CNNC) aims to classify nodes in a label-deficient target network by transferring the knowledge from a source network with abundant labels.
We propose a domain-adaptive message passing graph neural network (DM-GNN), which integrates graph neural network (GNN) with conditional adversarial domain adaptation.
arXiv Detail & Related papers (2023-08-31T05:26:08Z) - Feature Activation Map: Visual Explanation of Deep Learning Models for
Image Classification [17.373054348176932]
In this work, a post-hoc interpretation tool named feature activation map (FAM) is proposed.
FAM can interpret deep learning models without FC layers as a classifier.
Experiments conducted on ten deep learning models for few-shot image classification, contrastive learning image classification and image retrieval tasks demonstrate the effectiveness of the proposed FAM algorithm.
arXiv Detail & Related papers (2023-07-11T05:33:46Z) - SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and
Quasi-Planar Segmentation [53.83313235792596]
We present a new methodology for real-time semantic mapping from RGB-D sequences.
It combines a 2D neural network and a 3D network based on a SLAM system with 3D occupancy mapping.
Our system achieves state-of-the-art semantic mapping quality within 2D-3D networks-based systems.
arXiv Detail & Related papers (2023-06-28T22:36:44Z) - Cluster-CAM: Cluster-Weighted Visual Interpretation of CNNs' Decision in
Image Classification [12.971559051829658]
Cluster-CAM is an effective and efficient gradient-free CNN interpretation algorithm.
We propose an artful strategy to forge a cognition-base map and cognition-scissors from clustered feature maps.
arXiv Detail & Related papers (2023-02-03T10:38:20Z) - Opti-CAM: Optimizing saliency maps for interpretability [10.122899813335694]
We introduce Opti-CAM, combining ideas from CAM-based and masking-based approaches.
Our saliency map is a linear combination of feature maps, where weights are optimized per image.
On several datasets, Opti-CAM largely outperforms other CAM-based approaches according to the most relevant classification metrics.
arXiv Detail & Related papers (2023-01-17T16:44:48Z) - Learning Visual Explanations for DCNN-Based Image Classifiers Using an
Attention Mechanism [8.395400675921515]
Two new learning-based AI (XAI) methods for deep convolutional neural network (DCNN) image classifiers, called L-CAM-Fm and L-CAM-Img, are proposed.
Both methods use an attention mechanism that is inserted in the original (frozen) DCNN and is trained to derive class activation maps (CAMs) from the last convolutional layer's feature maps.
Experimental evaluation on ImageNet shows that the proposed methods achieve competitive results while requiring a single forward pass at the inference stage.
arXiv Detail & Related papers (2022-09-22T17:33:18Z) - Shap-CAM: Visual Explanations for Convolutional Neural Networks based on
Shapley Value [86.69600830581912]
We develop a novel visual explanation method called Shap-CAM based on class activation mapping.
We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process.
arXiv Detail & Related papers (2022-08-07T00:59:23Z) - Towards Learning Spatially Discriminative Feature Representations [26.554140976236052]
We propose a novel loss function, termed as CAM-loss, to constrain the embedded feature maps with the class activation maps (CAMs)
CAM-loss drives the backbone to express the features of target category and suppress the features of non-target categories or background.
Experimental results show that CAM-loss is applicable to a variety of network structures and can be combined with mainstream regularization methods to improve the performance of image classification.
arXiv Detail & Related papers (2021-09-03T08:04:17Z) - TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised
Object Localization [112.46381729542658]
Weakly supervised object localization (WSOL) is a challenging problem when given image category labels.
We introduce the token semantic coupled attention map (TS-CAM) to take full advantage of the self-attention mechanism in visual transformer for long-range dependency extraction.
arXiv Detail & Related papers (2021-03-27T09:43:16Z) - Video-based Facial Expression Recognition using Graph Convolutional
Networks [57.980827038988735]
We introduce a Graph Convolutional Network (GCN) layer into a common CNN-RNN based model for video-based facial expression recognition.
We evaluate our method on three widely-used datasets, CK+, Oulu-CASIA and MMI, and also one challenging wild dataset AFEW8.0.
arXiv Detail & Related papers (2020-10-26T07:31:51Z) - Towards Interpretable Semantic Segmentation via Gradient-weighted Class
Activation Mapping [71.91734471596432]
We propose SEG-GRAD-CAM, a gradient-based method for interpreting semantic segmentation.
Our method is an extension of the widely-used Grad-CAM method, applied locally to produce heatmaps showing the relevance of individual pixels for semantic segmentation.
arXiv Detail & Related papers (2020-02-26T12:32:40Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.