Contrastive Concept Importance: Explaining Pairwise Class Decisions Through Automatically Extracted Concept Representations

• URL: http://arxiv.org/abs/2607.27904v1
• Date: Thu, 30 Jul 2026 09:19:21 GMT
• Title: Contrastive Concept Importance: Explaining Pairwise Class Decisions Through Automatically Extracted Concept Representations
• Abstract summary: We introduce contrastive concept importance (CCI), which attributes the logit margin between a target class and a contrast, or foil, class to concepts.<n>We evaluate the method on ImageNet class pairs using CRAFT-style concept bases, insertion and deletion curves, logit-wise decomposition analysis, and semantic class hierarchy.
• Score: 6.466389033890039
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Concept-based explanations are a prevalent way to explain the decisions of complex black-box methods through semantically meaningful, human-interpretable concepts. To attribute the contribution of such concepts to a model's decisions, feature attribution methods are used to quantify how strongly each concept contributes to a model output. These attributions are typically computed for a single output class and therefore answer a non-contrastive "why P?" question. In many situations, however, such as cases of misclassification, class confusion, and low-margin predictions, the more natural question to ask is "why P rather than Q?". We introduce contrastive concept importance (CCI), which attributes the logit margin between a target class and a contrast, or foil, class to concepts in an automatically extracted visual concept basis. The resulting scores are signed, indicating whether a concept supports the target over the foil or the foil over the target, and can be decomposed into target-logit and foil-logit effects. This makes it possible to distinguish globally important concepts from concepts that specifically influence a class-pair distinction, including whether their effect is shared, one-sided, or directly contrastive. We evaluate the method on ImageNet class pairs using CRAFT-style concept bases, insertion and deletion curves, logit-wise decomposition analysis, and semantic class hierarchy. The results show that contrastive concept importance reveals class-pair-specific model behavior that is not captured by ordinary concept importance alone, and that highly contrastive concepts can be evaluated against semantic superclass structure to assess whether they affect fine-grained distinctions rather than broad category evidence.

Related papers

• Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition [81.2779530670268]
  Vision-Language Models (VLMs) have significantly advanced zero-shot image recognition.<n>This paper enhances prompt by incorporating class-specific concepts.<n>Our method consistently outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2026-03-09T03:11:11Z)
• Insight: Interpretable Semantic Hierarchies in Vision-Language Encoders [52.94006363830628]
  Language-aligned vision foundation models perform strongly across diverse downstream tasks.<n>Recent works decompose these representations into human-interpretable concepts, but provide poor spatial grounding and are limited to image classification tasks.<n>We propose Insight, a language-aligned concept foundation model that provides fine-grained concepts, which are human-interpretable and spatially grounded in the input image.
  arXiv  Detail & Related papers  (2026-01-20T09:57:26Z)
• FaCT: Faithful Concept Traces for Explaining Neural Network Decisions [56.796533084868884]
  Deep networks have shown remarkable performance across a wide range of tasks, yet getting a global concept-level understanding of how they function remains a key challenge.<n>We put emphasis on the faithfulness of concept-based explanations and propose a new model with model-inherent mechanistic concept-explanations.<n>Our concepts are shared across classes and, from any layer, their contribution to the logit and their input-visualization can be faithfully traced.
  arXiv  Detail & Related papers  (2025-10-29T13:35:46Z)
• Concept Bottleneck with Visual Concept Filtering for Explainable Medical Image Classification [16.849592713393896]
  Concept Bottleneck Models (CBMs) enable interpretable image classification by utilizing human-understandable concepts as intermediate targets. We propose a visual activation score that measures whether the concept contains visual cues or not. Computed visual activation scores are then used to filter out the less visible concepts, thus resulting in a final concept set with visually meaningful concepts.
  arXiv  Detail & Related papers  (2023-08-23T05:04:01Z)
• Statistically Significant Concept-based Explanation of Image Classifiers via Model Knockoffs [22.576922942465142]
  Concept-based explanations may cause false positives, which misregards unrelated concepts as important for the prediction task. We propose a method using a deep learning model to learn the image concept and then using the Knockoff samples to select the important concepts for prediction.
  arXiv  Detail & Related papers  (2023-05-27T05:40:05Z)
• Concept Gradient: Concept-based Interpretation Without Linear Assumption [77.96338722483226]
  Concept Activation Vector (CAV) relies on learning a linear relation between some latent representation of a given model and concepts. We proposed Concept Gradient (CG), extending concept-based interpretation beyond linear concept functions. We demonstrated CG outperforms CAV in both toy examples and real world datasets.
  arXiv  Detail & Related papers  (2022-08-31T17:06:46Z)
• Translational Concept Embedding for Generalized Compositional Zero-shot Learning [73.60639796305415]
  Generalized compositional zero-shot learning means to learn composed concepts of attribute-object pairs in a zero-shot fashion. This paper introduces a new approach, termed translational concept embedding, to solve these two difficulties in a unified framework.
  arXiv  Detail & Related papers  (2021-12-20T21:27:51Z)
• Is Disentanglement all you need? Comparing Concept-based & Disentanglement Approaches [24.786152654589067]
  We give an overview of concept-based explanations and disentanglement approaches. We show that state-of-the-art approaches from both classes can be data inefficient, sensitive to the specific nature of the classification/regression task, or sensitive to the employed concept representation.
  arXiv  Detail & Related papers  (2021-04-14T15:06:34Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• CURI: A Benchmark for Productive Concept Learning Under Uncertainty [33.83721664338612]
  We introduce a new few-shot, meta-learning benchmark, Compositional Reasoning Under Uncertainty (CURI) CURI evaluates different aspects of productive and systematic generalization, including abstract understandings of disentangling, productive generalization, learning operations, variable binding, etc. It also defines a model-independent "compositionality gap" to evaluate the difficulty of generalizing out-of-distribution along each of these axes.
  arXiv  Detail & Related papers  (2020-10-06T16:23:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.