Towards a Deeper Understanding of Concept Bottleneck Models Through End-to-End Explanation

• URL: http://arxiv.org/abs/2302.03578v1
• Date: Tue, 7 Feb 2023 16:43:43 GMT
• Title: Towards a Deeper Understanding of Concept Bottleneck Models Through End-to-End Explanation
• Authors: Jack Furby, Daniel Cunnington, Dave Braines, Alun Preece
• Abstract summary: Concept Bottleneck Models (CBMs) first map raw input(s) to a vector of human-defined concepts, before using this vector to predict a final classification. In doing so, this would support human interpretation when generating explanations of the model's outputs to visualise input features corresponding to concepts.
• Score: 2.9740255333669454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Concept Bottleneck Models (CBMs) first map raw input(s) to a vector of human-defined concepts, before using this vector to predict a final classification. We might therefore expect CBMs capable of predicting concepts based on distinct regions of an input. In doing so, this would support human interpretation when generating explanations of the model's outputs to visualise input features corresponding to concepts. The contribution of this paper is threefold: Firstly, we expand on existing literature by looking at relevance both from the input to the concept vector, confirming that relevance is distributed among the input features, and from the concept vector to the final classification where, for the most part, the final classification is made using concepts predicted as present. Secondly, we report a quantitative evaluation to measure the distance between the maximum input feature relevance and the ground truth location; we perform this with the techniques, Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG) and a baseline gradient approach, finding LRP has a lower average distance than IG. Thirdly, we propose using the proportion of relevance as a measurement for explaining concept importance.

Related papers

• Interpretable Hierarchical Concept Reasoning through Attention-Guided Graph Learning [8.464865102100925]
  We propose Hierarchical Concept Memory Reasoner (H-CMR) to provide interpretability for both concept and task predictions.<n>H-CMR matches state-of-the-art performance while enabling strong human interaction through concept and model interventions.
  arXiv  Detail & Related papers  (2025-06-26T08:56:55Z)
• I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data? [76.15163242945813]
  Large language models (LLMs) have led many to conclude that they exhibit a form of intelligence.<n>We introduce a novel generative model that generates tokens on the basis of human-interpretable concepts represented as latent discrete variables.
  arXiv  Detail & Related papers  (2025-03-12T01:21:17Z)
• Concept Boundary Vectors [0.0]
  We introduce concept boundary vectors as a concept vector construction derived from the boundary between the latent representations of concepts. Empirically we demonstrate that concept boundary vectors capture a concept's semantic meaning, and we compare their effectiveness against concept activation vectors.
  arXiv  Detail & Related papers  (2024-12-20T09:18:11Z)
• Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery [52.498055901649025]
  Concept Bottleneck Models (CBMs) have been proposed to address the 'black-box' problem of deep neural networks. We propose a novel CBM approach -- called Discover-then-Name-CBM (DN-CBM) -- that inverts the typical paradigm. Our concept extraction strategy is efficient, since it is agnostic to the downstream task, and uses concepts already known to the model.
  arXiv  Detail & Related papers  (2024-07-19T17:50:11Z)
• On the Concept Trustworthiness in Concept Bottleneck Models [39.928868605678744]
  Concept Bottleneck Models (CBMs) break down the reasoning process into the input-to-concept mapping and the concept-to-label prediction. Despite the transparency of the concept-to-label prediction, the mapping from the input to the intermediate concept remains a black box. A pioneering metric, referred to as concept trustworthiness score, is proposed to gauge whether the concepts are derived from relevant regions. An enhanced CBM is introduced, enabling concept predictions to be made specifically from distinct parts of the feature map.
  arXiv  Detail & Related papers  (2024-03-21T12:24:53Z)
• Can we Constrain Concept Bottleneck Models to Learn Semantically Meaningful Input Features? [0.6401548653313325]
  Concept Bottleneck Models (CBMs) are regarded as inherently interpretable because they first predict a set of human-defined concepts. Current literature suggests that concept predictions often rely on irrelevant input features. In this paper, we demonstrate that CBMs can learn to map concepts to semantically meaningful input features.
  arXiv  Detail & Related papers  (2024-02-01T10:18:43Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Generalizing Backpropagation for Gradient-Based Interpretability [103.2998254573497]
  We show that the gradient of a model is a special case of a more general formulation using semirings. This observation allows us to generalize the backpropagation algorithm to efficiently compute other interpretable statistics.
  arXiv  Detail & Related papers  (2023-07-06T15:19:53Z)
• Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanations [14.77637281844823]
  We propose the post-hoc eXplainable Artificial Intelligence method L-CRP to generate explanations that automatically identify and visualize relevant concepts learned, recognized and used by the model during inference as well as precisely locate them in input space. We verify the faithfulness of our proposed technique by quantitatively comparing different concept attribution methods, and discuss the effect on explanation complexity on popular datasets such as CityScapes, Pascal VOC and MS COCO 2017.
  arXiv  Detail & Related papers  (2022-11-21T13:12:23Z)
• Concept Gradient: Concept-based Interpretation Without Linear Assumption [77.96338722483226]
  Concept Activation Vector (CAV) relies on learning a linear relation between some latent representation of a given model and concepts. We proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We demonstrated CG outperforms CAV in both toy examples and real world datasets.
  arXiv  Detail & Related papers  (2022-08-31T17:06:46Z)
• Exploring Concept Contribution Spatially: Hidden Layer Interpretation with Spatial Activation Concept Vector [5.873416857161077]
  Testing with Concept Activation Vector (TCAV) presents a powerful tool to quantify the contribution of query concepts to a target class. For some images where the target object only occupies a small fraction of the region, TCAV evaluation may be interfered with by redundant background features.
  arXiv  Detail & Related papers  (2022-05-21T15:58:57Z)
• Improving Aspect-based Sentiment Analysis with Gated Graph Convolutional Networks and Syntax-based Regulation [89.38054401427173]
  Aspect-based Sentiment Analysis (ABSA) seeks to predict the sentiment polarity of a sentence toward a specific aspect. dependency trees can be integrated into deep learning models to produce the state-of-the-art performance for ABSA. We propose a novel graph-based deep learning model to overcome these two issues.
  arXiv  Detail & Related papers  (2020-10-26T07:36:24Z)
• Medical Concept Normalization in User Generated Texts by Learning Target Concept Embeddings [5.33024001730262]
  Recent research approach concept normalization as either text classification or text matching. Our proposed model overcomes these drawbacks by jointly learning the representations of input concept mention and target concepts. Our model surpasses all the existing methods across three standard datasets by improving accuracy up to 2.31%.
  arXiv  Detail & Related papers  (2020-06-07T01:17:18Z)
• R-VGAE: Relational-variational Graph Autoencoder for Unsupervised Prerequisite Chain Learning [83.13634692459486]
  We propose a model called Graph AutoEncoder (VGA-E) to predict concept relations within a graph consisting of concept resource nodes. Results show that our unsupervised approach outperforms graph-based semi-supervised methods and other baseline methods by up to 9.77% and 10.47% in terms of prerequisite relation prediction accuracy and F1 score. Our method is notably the first graph-based model that attempts to make use of deep learning representations for the task of unsupervised prerequisite learning.
  arXiv  Detail & Related papers  (2020-04-22T14:48:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.