GCPV: Guided Concept Projection Vectors for the Explainable Inspection
of CNN Feature Spaces
- URL: http://arxiv.org/abs/2311.14435v1
- Date: Fri, 24 Nov 2023 12:22:00 GMT
- Title: GCPV: Guided Concept Projection Vectors for the Explainable Inspection
of CNN Feature Spaces
- Authors: Georgii Mikriukov, Gesina Schwalbe, Christian Hellert, Korinna Bade
- Abstract summary: We introduce the local-to-global Guided Concept Projection Vectors (GCPV) approach.
GCPV generates local concept vectors that each precisely reconstruct a concept segmentation label.
It then generalizes these to global concept and even sub-concept vectors by means of hiearchical clustering.
- Score: 1.0923877073891446
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: For debugging and verification of computer vision convolutional deep neural
networks (CNNs) human inspection of the learned latent representations is
imperative. Therefore, state-of-the-art eXplainable Artificial Intelligence
(XAI) methods globally associate given natural language semantic concepts with
representing vectors or regions in the CNN latent space supporting manual
inspection. Yet, this approach comes with two major disadvantages: They are
locally inaccurate when reconstructing a concept label and discard information
about the distribution of concept instance representations. The latter, though,
is of particular interest for debugging, like finding and understanding
outliers, learned notions of sub-concepts, and concept confusion. Furthermore,
current single-layer approaches neglect that information about a concept may be
spread over the CNN depth. To overcome these shortcomings, we introduce the
local-to-global Guided Concept Projection Vectors (GCPV) approach: It (1)
generates local concept vectors that each precisely reconstruct a concept
segmentation label, and then (2) generalizes these to global concept and even
sub-concept vectors by means of hiearchical clustering. Our experiments on
object detectors demonstrate improved performance compared to the
state-of-the-art, the benefit of multi-layer concept vectors, and robustness
against low-quality concept segmentation labels. Finally, we demonstrate that
GCPVs can be applied to find root causes for confusion of concepts like bus and
truck, and reveal interesting concept-level outliers. Thus, GCPVs pose a
promising step towards interpretable model debugging and informed data
improvement.
Related papers
- Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery [52.498055901649025]
Concept Bottleneck Models (CBMs) have been proposed to address the 'black-box' problem of deep neural networks.
We propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm.
Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model.
arXiv Detail & Related papers (2024-07-19T17:50:11Z) - Locally Testing Model Detections for Semantic Global Concepts [3.112979958793927]
We propose a framework for linking global concept encodings to the local processing of single network inputs.
Our approach has the advantage of fully covering the model-internal encoding of the semantic concept.
The results show major differences in the local perception and usage of individual global concept encodings.
arXiv Detail & Related papers (2024-05-27T12:52:45Z) - Unsupervised Interpretable Basis Extraction for Concept-Based Visual
Explanations [53.973055975918655]
We show that, intermediate layer representations become more interpretable when transformed to the bases extracted with our method.
We compare the bases extracted with our method with the bases derived with a supervised approach and find that, in one aspect, the proposed unsupervised approach has a strength that constitutes a limitation of the supervised one and give potential directions for future research.
arXiv Detail & Related papers (2023-03-19T00:37:19Z) - Concept Activation Regions: A Generalized Framework For Concept-Based
Explanations [95.94432031144716]
Existing methods assume that the examples illustrating a concept are mapped in a fixed direction of the deep neural network's latent space.
In this work, we propose allowing concept examples to be scattered across different clusters in the DNN's latent space.
This concept activation region (CAR) formalism yields global concept-based explanations and local concept-based feature importance.
arXiv Detail & Related papers (2022-09-22T17:59:03Z) - Visual Recognition with Deep Nearest Centroids [57.35144702563746]
We devise deep nearest centroids (DNC), a conceptually elegant yet surprisingly effective network for large-scale visual recognition.
Compared with parametric counterparts, DNC performs better on image classification (CIFAR-10, ImageNet) and greatly boots pixel recognition (ADE20K, Cityscapes)
arXiv Detail & Related papers (2022-09-15T15:47:31Z) - Concept Gradient: Concept-based Interpretation Without Linear Assumption [77.96338722483226]
Concept Activation Vector (CAV) relies on learning a linear relation between some latent representation of a given model and concepts.
We proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions.
We demonstrated CG outperforms CAV in both toy examples and real world datasets.
arXiv Detail & Related papers (2022-08-31T17:06:46Z) - GlanceNets: Interpretabile, Leak-proof Concept-based Models [23.7625973884849]
Concept-based models (CBMs) combine high-performance and interpretability by acquiring and reasoning with a vocabulary of high-level concepts.
We provide a clear definition of interpretability in terms of alignment between the model's representation and an underlying data generation process.
We introduce GlanceNets, a new CBM that exploits techniques from disentangled representation learning and open-set recognition to achieve alignment.
arXiv Detail & Related papers (2022-05-31T08:53:53Z) - Exploring Concept Contribution Spatially: Hidden Layer Interpretation
with Spatial Activation Concept Vector [5.873416857161077]
Testing with Concept Activation Vector (TCAV) presents a powerful tool to quantify the contribution of query concepts to a target class.
For some images where the target object only occupies a small fraction of the region, TCAV evaluation may be interfered with by redundant background features.
arXiv Detail & Related papers (2022-05-21T15:58:57Z) - SegDiscover: Visual Concept Discovery via Unsupervised Semantic
Segmentation [29.809900593362844]
SegDiscover is a novel framework that discovers semantically meaningful visual concepts from imagery datasets with complex scenes without supervision.
Our method generates concept primitives from raw images, discovering concepts by clustering in the latent space of a self-supervised pretrained encoder, and concept refinement via neural network smoothing.
arXiv Detail & Related papers (2022-04-22T20:44:42Z) - Human-Centered Concept Explanations for Neural Networks [47.71169918421306]
We introduce concept explanations including the class of Concept Activation Vectors (CAV)
We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats.
Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.
arXiv Detail & Related papers (2022-02-25T01:27:31Z) - Invertible Concept-based Explanations for CNN Models with Non-negative
Concept Activation Vectors [24.581839689833572]
Convolutional neural network (CNN) models for computer vision are powerful but lack explainability in their most basic form.
Recent work on explanations through feature importance of approximate linear models has moved from input-level features to features from mid-layer feature maps in the form of concept activation vectors (CAVs)
In this work, we rethink the ACE algorithm of Ghorbani etal., proposing an alternative invertible concept-based explanation (ICE) framework to overcome its shortcomings.
arXiv Detail & Related papers (2020-06-27T17:57:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.