Towards Spatially-Aware and Optimally Faithful Concept-Based Explanations

• URL: http://arxiv.org/abs/2504.10833v1
• Date: Tue, 15 Apr 2025 03:24:13 GMT
• Title: Towards Spatially-Aware and Optimally Faithful Concept-Based Explanations
• Authors: Shubham Kumar, Dwip Dalal, Narendra Ahuja,
• Abstract summary: Post-hoc, unsupervised concept-based explanation methods (U-CBEMs) are a promising tool for generating semantic explanations of the decision-making processes in deep neural networks.<n>We identify several limitations of prior faithfulness metrics that inhibit an accurate evaluation.<n>We address these limitations with Surrogate Faithfulness (SF), an evaluation method that introduces a spatially-aware surrogate and two novel faithfulness metrics.
• Score: 9.34674232134837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-hoc, unsupervised concept-based explanation methods (U-CBEMs) are a promising tool for generating semantic explanations of the decision-making processes in deep neural networks, having applications in both model improvement and understanding. It is vital that the explanation is accurate, or faithful, to the model, yet we identify several limitations of prior faithfulness metrics that inhibit an accurate evaluation; most notably, prior metrics involve only the set of concepts present, ignoring how they may be spatially distributed. We address these limitations with Surrogate Faithfulness (SF), an evaluation method that introduces a spatially-aware surrogate and two novel faithfulness metrics. Using SF, we produce Optimally Faithful (OF) explanations, where concepts are found that maximize faithfulness. Our experiments show that (1) adding spatial-awareness to prior U-CBEMs increases faithfulness in all cases; (2) OF produces significantly more faithful explanations than prior U-CBEMs (30% or higher improvement in error); (3) OF's learned concepts generalize well to out-of-domain data and are more robust to adversarial examples, where prior U-CBEMs struggle.

Related papers

• Towards Robust and Reliable Concept Representations: Reliability-Enhanced Concept Embedding Model [22.865870813626316]
  Concept Bottleneck Models (CBMs) aim to enhance interpretability by predicting human-understandable concepts as intermediates for decision-making. Two inherent issues contribute to concept unreliability: sensitivity to concept-irrelevant features and lack of semantic consistency for the same concept across different samples. We propose the Reliability-Enhanced Concept Embedding Model (RECEM), which introduces a two-fold strategy: Concept-Level Disentanglement to separate irrelevant features from concept-relevant information and a Concept Mixup mechanism to ensure semantic alignment across samples.
  arXiv  Detail & Related papers  (2025-02-03T09:29:39Z)
• Concept-Based Explainable Artificial Intelligence: Metrics and Benchmarks [0.0]
  Concept-based explanation methods aim to improve the interpretability of machine learning models.<n>We propose three metrics: the concept global importance metric, the concept existence metric, and the concept location metric.<n>We demonstrate that, in many cases, even the most important concepts determined by post-hoc CBMs are not present in input images.
  arXiv  Detail & Related papers  (2025-01-31T16:32:36Z)
• Concept-driven Off Policy Evaluation [2.789652596206117]
  We develop a family of concept-based OPE estimators, proving that they remain unbiased and reduce variance when concepts are known and predefined.<n>Experiments with synthetic and real-world datasets show that both known and learned concept-based estimators significantly improve OPE performance.<n>Unlike other OPE methods, concept-based estimators are easily interpretable and allow for targeted interventions on specific concepts, further enhancing the quality of these estimators.
  arXiv  Detail & Related papers  (2024-11-28T22:15:06Z)
• MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction [57.483718822429346]
  MulCPred is proposed that explains its predictions based on multi-modal concepts represented by training samples. MulCPred is evaluated on multiple datasets and tasks.
  arXiv  Detail & Related papers  (2024-09-14T14:15:28Z)
• Rethinking State Disentanglement in Causal Reinforcement Learning [78.12976579620165]
  Causality provides rigorous theoretical support for ensuring that the underlying states can be uniquely recovered through identifiability. We revisit this research line and find that incorporating RL-specific context can reduce unnecessary assumptions in previous identifiability analyses for latent states. We propose a novel approach for general partially observable Markov Decision Processes (POMDPs) by replacing the complicated structural constraints in previous methods with two simple constraints for transition and reward preservation.
  arXiv  Detail & Related papers  (2024-08-24T06:49:13Z)
• Evaluating Readability and Faithfulness of Concept-based Explanations [35.48852504832633]
  Concept-based explanations arise as a promising avenue for explaining high-level patterns learned by Large Language Models. Current methods approach concepts from different perspectives, lacking a unified formalization. This makes evaluating the core measures of concepts, namely faithfulness or readability, challenging.
  arXiv  Detail & Related papers  (2024-04-29T09:20:25Z)
• Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals [67.64770842323966]
  Causal explanations of predictions of NLP systems are essential to ensure safety and establish trust. Existing methods often fall short of explaining model predictions effectively or efficiently. We propose two approaches for counterfactual (CF) approximation.
  arXiv  Detail & Related papers  (2023-10-01T07:31:04Z)
• DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications [54.93807822347193]
  We show how to adapt attribution robustness estimation methods to a given domain, so as to take into account domain-specific plausibility. Next, we provide two methods, adversarial training and FAR training, to mitigate the brittleness characterized by DARE. Finally, we empirically validate our methods with extensive experiments on three established biomedical benchmarks.
  arXiv  Detail & Related papers  (2023-07-05T08:11:40Z)
• Networked Communication for Decentralised Agents in Mean-Field Games [59.01527054553122]
  We introduce networked communication to the mean-field game framework.<n>We prove that our architecture has sample guarantees bounded between those of the centralised- and independent-learning cases.<n>We show that our networked approach has significant advantages over both alternatives in terms of robustness to update failures and to changes in population size.
  arXiv  Detail & Related papers  (2023-06-05T10:45:39Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Entity Concept-enhanced Few-shot Relation Extraction [35.10974511223129]
  Few-shot relation extraction (FSRE) is of great importance in long-tail distribution problem. Most existing FSRE algorithms fail to accurately classify the relations merely based on the information of the sentences together with the recognized entity pairs. We propose a novel entity-enhanced FEw-shot Relation Extraction scheme (ConceptFERE), which introduces the inherent concepts of entities to provide clues for relation prediction.
  arXiv  Detail & Related papers  (2021-06-04T10:36:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.