Concept Gradient: Concept-based Interpretation Without Linear Assumption

• URL: http://arxiv.org/abs/2208.14966v2
• Date: Mon, 5 Feb 2024 21:27:45 GMT
• Title: Concept Gradient: Concept-based Interpretation Without Linear Assumption
• Authors: Andrew Bai, Chih-Kuan Yeh, Pradeep Ravikumar, Neil Y. C. Lin, Cho-Jui Hsieh
• Abstract summary: Concept Activation Vector (CAV) relies on learning a linear relation between some latent representation of a given model and concepts. We proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We demonstrated CG outperforms CAV in both toy examples and real world datasets.
• Score: 77.96338722483226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Concept-based interpretations of black-box models are often more intuitive for humans to understand. The most widely adopted approach for concept-based interpretation is Concept Activation Vector (CAV). CAV relies on learning a linear relation between some latent representation of a given model and concepts. The linear separability is usually implicitly assumed but does not hold true in general. In this work, we started from the original intent of concept-based interpretation and proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We showed that for a general (potentially non-linear) concept, we can mathematically evaluate how a small change of concept affecting the model's prediction, which leads to an extension of gradient-based interpretation to the concept space. We demonstrated empirically that CG outperforms CAV in both toy examples and real world datasets.

Related papers

• Improving Intervention Efficacy via Concept Realignment in Concept Bottleneck Models [57.86303579812877]
  Concept Bottleneck Models (CBMs) ground image classification on human-understandable concepts to allow for interpretable model decisions. Existing approaches often require numerous human interventions per image to achieve strong performances. We introduce a trainable concept realignment intervention module, which leverages concept relations to realign concept assignments post-intervention.
  arXiv  Detail & Related papers  (2024-05-02T17:59:01Z)
• Knowledge graphs for empirical concept retrieval [1.06378109904813]
  Concept-based explainable AI is promising as a tool to improve the understanding of complex models at the premises of a given user. Here, we present a workflow for user-driven data collection in both text and image domains. We test the retrieved concept datasets on two concept-based explainability methods, namely concept activation vectors (CAVs) and concept activation regions (CARs)
  arXiv  Detail & Related papers  (2024-04-10T13:47:22Z)
• An Axiomatic Approach to Model-Agnostic Concept Explanations [67.84000759813435]
  We propose an approach to concept explanations that satisfy three natural axioms: linearity, recursivity, and similarity. We then establish connections with previous concept explanation methods, offering insight into their varying semantic meanings.
  arXiv  Detail & Related papers  (2024-01-12T20:53:35Z)
• Identifying Linear Relational Concepts in Large Language Models [16.917379272022064]
  Transformer language models (LMs) have been shown to represent concepts as directions in the latent space of hidden activations. We present a technique called linear relational concepts (LRC) for finding concept directions corresponding to human-interpretable concepts.
  arXiv  Detail & Related papers  (2023-11-15T14:01:41Z)
• Interpretable Neural-Symbolic Concept Reasoning [7.1904050674791185]
  Concept-based models aim to address this issue by learning tasks based on a set of human-understandable concepts. We propose the Deep Concept Reasoner (DCR), the first interpretable concept-based model that builds upon concept embeddings.
  arXiv  Detail & Related papers  (2023-04-27T09:58:15Z)
• Concept Activation Regions: A Generalized Framework For Concept-Based Explanations [95.94432031144716]
  Existing methods assume that the examples illustrating a concept are mapped in a fixed direction of the deep neural network's latent space. In this work, we propose allowing concept examples to be scattered across different clusters in the DNN's latent space. This concept activation region (CAR) formalism yields global concept-based explanations and local concept-based feature importance.
  arXiv  Detail & Related papers  (2022-09-22T17:59:03Z)
• GlanceNets: Interpretabile, Leak-proof Concept-based Models [23.7625973884849]
  Concept-based models (CBMs) combine high-performance and interpretability by acquiring and reasoning with a vocabulary of high-level concepts. We provide a clear definition of interpretability in terms of alignment between the model's representation and an underlying data generation process. We introduce GlanceNets, a new CBM that exploits techniques from disentangled representation learning and open-set recognition to achieve alignment.
  arXiv  Detail & Related papers  (2022-05-31T08:53:53Z)
• ConceptDistil: Model-Agnostic Distillation of Concept Explanations [4.462334751640166]
  Concept-based explanations aims to fill the model interpretability gap for non-technical humans-in-the-loop. We propose ConceptDistil, a method to bring concept explanations to any black-box classifier using knowledge distillation. We validate ConceptDistil in a real world use-case, showing that it is able to optimize both tasks.
  arXiv  Detail & Related papers  (2022-05-07T08:58:54Z)
• Human-Centered Concept Explanations for Neural Networks [47.71169918421306]
  We introduce concept explanations including the class of Concept Activation Vectors (CAV) We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats. Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.
  arXiv  Detail & Related papers  (2022-02-25T01:27:31Z)
• Kernelized Concept Erasure [108.65038124096907]
  We propose a kernelization of a linear minimax game for concept erasure. It is possible to prevent specific non-linear adversaries from predicting the concept. However, the protection does not transfer to different nonlinear adversaries.
  arXiv  Detail & Related papers  (2022-01-28T15:45:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.