Towards Spatially-Aware and Optimally Faithful Concept-Based Explanations

• URL: http://arxiv.org/abs/2504.10833v1
• Date: Tue, 15 Apr 2025 03:24:13 GMT
• Title: Towards Spatially-Aware and Optimally Faithful Concept-Based Explanations
• Abstract summary: Post-hoc, unsupervised concept-based explanation methods (U-CBEMs) are a promising tool for generating semantic explanations of the decision-making processes in deep neural networks.<n>We identify several limitations of prior faithfulness metrics that inhibit an accurate evaluation.<n>We address these limitations with Surrogate Faithfulness (SF), an evaluation method that introduces a spatially-aware surrogate and two novel faithfulness metrics.
• Score: 9.34674232134837
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Post-hoc, unsupervised concept-based explanation methods (U-CBEMs) are a promising tool for generating semantic explanations of the decision-making processes in deep neural networks, having applications in both model improvement and understanding. It is vital that the explanation is accurate, or faithful, to the model, yet we identify several limitations of prior faithfulness metrics that inhibit an accurate evaluation; most notably, prior metrics involve only the set of concepts present, ignoring how they may be spatially distributed. We address these limitations with Surrogate Faithfulness (SF), an evaluation method that introduces a spatially-aware surrogate and two novel faithfulness metrics. Using SF, we produce Optimally Faithful (OF) explanations, where concepts are found that maximize faithfulness. Our experiments show that (1) adding spatial-awareness to prior U-CBEMs increases faithfulness in all cases; (2) OF produces significantly more faithful explanations than prior U-CBEMs (30% or higher improvement in error); (3) OF's learned concepts generalize well to out-of-domain data and are more robust to adversarial examples, where prior U-CBEMs struggle.

Related papers

• ReCBM: Uncertainty-Gated Relational Reasoning for Concept Bottleneck Models [16.64288013782432]
  Concept Bottleneck Models (CBMs) provide an interpretable framework by grounding predictions in human-understandable concepts.<n>Recent variants have improved CBMs through richer concept representations, uncertainty estimation, and dependency modeling.<n>We propose ReCBM, an uncertainty-gated relational reasoning framework for CBMs.
  arXiv  Detail & Related papers  (2026-08-07T19:59:47Z)
• Faithfulness Serum: Mitigating the Faithfulness Gap in Textual Explanations of LLM Decisions via Attribution Guidance [57.17102098930037]
  Large language models (LLMs) achieve strong performance and have revolutionized NLP.<n>Their lack of explainability keeps them treated as black boxes, limiting their use in domains that demand transparency and trust.<n>We introduce a training-free method that enhances faithfulness by guiding explanation generation through attention-level interventions.
  arXiv  Detail & Related papers  (2026-04-15T18:32:32Z)
• Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing [5.497064111008921]
  We propose textbfSelf-textbfAudited textbfVerified textbfReasoning (textscSAVeR), a novel framework that enforces verification over internal belief states within the agent before action commitment.<n>Our approach consistently improves reasoning faithfulness while preserving competitive end-task performance.
  arXiv  Detail & Related papers  (2026-04-09T16:01:03Z)
• Uncertainty-aware Language Guidance for Concept Bottleneck Models [19.882022420045804]
  Concept Bottleneck Models (CBMs) provide inherent interpretability by first mapping input samples to high-level semantic concepts, followed by a combination of these concepts for the final classification.<n>However, the annotation of human-understandable concepts requires extensive expert knowledge and labor, constraining the broad adoption of CBMs.<n>We propose a novel uncertainty-aware CBM method, which not only rigorously quantifies the uncertainty of LLM-annotated concept labels with valid and distribution-free guarantees, but also incorporates quantified concept uncertainty into the CBM training procedure.
  arXiv  Detail & Related papers  (2026-02-26T20:59:03Z)
• Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinking [11.763473690046721]
  Reasoning-augmented vision language models generate explicit chains of thought that promise greater capability and transparency.<n>Models may reach correct answers via visually unfaithful intermediate steps, or reason faithfully yet fail on the final prediction.<n>We introduce the visual faithfulness of reasoning chains as a distinct evaluation dimension, focusing on whether the perception steps of a reasoning chain are grounded in the image.
  arXiv  Detail & Related papers  (2025-12-13T07:04:42Z)
• The Illusion of Certainty: Uncertainty quantification for LLMs fails under ambiguity [48.899855816199484]
  We introduce MAQA* and AmbigQA*, the first ambiguous question-answering (QA) datasets equipped with ground-truth answer distributions.<n>We show that predictive-distribution and ensemble-based estimators are fundamentally limited under ambiguity.
  arXiv  Detail & Related papers  (2025-11-06T14:46:35Z)
• Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check [60.77691669644931]
  We propose Functional Alignment for Distributional Equivalence (FADE), a novel metric that measures distributional similarity between unlearned and reference models.<n>We show that FADE captures functional alignment across the entire output distribution, providing a principled assessment of genuine unlearning.<n>These findings expose fundamental gaps in current evaluation practices and demonstrate that FADE provides a more robust foundation for developing and assessing truly effective unlearning methods.
  arXiv  Detail & Related papers  (2025-10-14T20:50:30Z)
• FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning [62.452350134196934]
  FaithCoT-Bench is a unified benchmark for instance-level CoT unfaithfulness detection.<n>Our framework formulates unfaithfulness detection as a discriminative decision problem.<n>FaithCoT-Bench sets a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.
  arXiv  Detail & Related papers  (2025-10-05T05:16:54Z)
• Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space [62.54887038032942]
  We introduce Soft Thinking, a training-free method that emulates human-like "soft" reasoning by generating soft, abstract concept tokens.<n>These concept tokens are created by the probability-weighted mixture of token embeddings, which form the continuous concept space.<n>In essence, each generated concept token encapsulates multiple meanings from related discrete tokens, implicitly exploring various reasoning paths to converge.
  arXiv  Detail & Related papers  (2025-05-21T17:29:15Z)
• I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data? [76.15163242945813]
  Large language models (LLMs) have led many to conclude that they exhibit a form of intelligence.<n>We introduce a novel generative model that generates tokens on the basis of human-interpretable concepts represented as latent discrete variables.
  arXiv  Detail & Related papers  (2025-03-12T01:21:17Z)
• Towards Robust and Reliable Concept Representations: Reliability-Enhanced Concept Embedding Model [22.865870813626316]
  Concept Bottleneck Models (CBMs) aim to enhance interpretability by predicting human-understandable concepts as intermediates for decision-making. Two inherent issues contribute to concept unreliability: sensitivity to concept-irrelevant features and lack of semantic consistency for the same concept across different samples. We propose the Reliability-Enhanced Concept Embedding Model (RECEM), which introduces a two-fold strategy: Concept-Level Disentanglement to separate irrelevant features from concept-relevant information and a Concept Mixup mechanism to ensure semantic alignment across samples.
  arXiv  Detail & Related papers  (2025-02-03T09:29:39Z)
• Concept-Based Explainable Artificial Intelligence: Metrics and Benchmarks [0.0]
  Concept-based explanation methods aim to improve the interpretability of machine learning models.<n>We propose three metrics: the concept global importance metric, the concept existence metric, and the concept location metric.<n>We demonstrate that, in many cases, even the most important concepts determined by post-hoc CBMs are not present in input images.
  arXiv  Detail & Related papers  (2025-01-31T16:32:36Z)
• Concept-driven Off Policy Evaluation [2.789652596206117]
  We develop a family of concept-based OPE estimators, proving that they remain unbiased and reduce variance when concepts are known and predefined.<n>Experiments with synthetic and real-world datasets show that both known and learned concept-based estimators significantly improve OPE performance.<n>Unlike other OPE methods, concept-based estimators are easily interpretable and allow for targeted interventions on specific concepts, further enhancing the quality of these estimators.
  arXiv  Detail & Related papers  (2024-11-28T22:15:06Z)
• MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction [57.483718822429346]
  MulCPred is proposed that explains its predictions based on multi-modal concepts represented by training samples. MulCPred is evaluated on multiple datasets and tasks.
  arXiv  Detail & Related papers  (2024-09-14T14:15:28Z)
• Rethinking State Disentanglement in Causal Reinforcement Learning [78.12976579620165]
  Causality provides rigorous theoretical support for ensuring that the underlying states can be uniquely recovered through identifiability. We revisit this research line and find that incorporating RL-specific context can reduce unnecessary assumptions in previous identifiability analyses for latent states. We propose a novel approach for general partially observable Markov Decision Processes (POMDPs) by replacing the complicated structural constraints in previous methods with two simple constraints for transition and reward preservation.
  arXiv  Detail & Related papers  (2024-08-24T06:49:13Z)
• Cycles of Thought: Measuring LLM Confidence through Stable Explanations [53.15438489398938]
  Large language models (LLMs) can reach and even surpass human-level accuracy on a variety of benchmarks, but their overconfidence in incorrect responses is still a well-documented failure mode. We propose a framework for measuring an LLM's uncertainty with respect to the distribution of generated explanations for an answer.
  arXiv  Detail & Related papers  (2024-06-05T16:35:30Z)
• Evaluating Readability and Faithfulness of Concept-based Explanations [35.48852504832633]
  Concept-based explanations arise as a promising avenue for explaining high-level patterns learned by Large Language Models. Current methods approach concepts from different perspectives, lacking a unified formalization. This makes evaluating the core measures of concepts, namely faithfulness or readability, challenging.
  arXiv  Detail & Related papers  (2024-04-29T09:20:25Z)
• Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals [67.64770842323966]
  Causal explanations of predictions of NLP systems are essential to ensure safety and establish trust. Existing methods often fall short of explaining model predictions effectively or efficiently. We propose two approaches for counterfactual (CF) approximation.
  arXiv  Detail & Related papers  (2023-10-01T07:31:04Z)
• Goodhart's Law Applies to NLP's Explanation Benchmarks [57.26445915212884]
  We critically examine two sets of metrics: the ERASER metrics (comprehensiveness and sufficiency) and the EVAL-X metrics. We show that we can inflate a model's comprehensiveness and sufficiency scores dramatically without altering its predictions or explanations on in-distribution test inputs. Our results raise doubts about the ability of current metrics to guide explainability research, underscoring the need for a broader reassessment of what precisely these metrics are intended to capture.
  arXiv  Detail & Related papers  (2023-08-28T03:03:03Z)
• DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applications [54.93807822347193]
  We show how to adapt attribution robustness estimation methods to a given domain, so as to take into account domain-specific plausibility. Next, we provide two methods, adversarial training and FAR training, to mitigate the brittleness characterized by DARE. Finally, we empirically validate our methods with extensive experiments on three established biomedical benchmarks.
  arXiv  Detail & Related papers  (2023-07-05T08:11:40Z)
• Networked Communication for Decentralised Agents in Mean-Field Games [59.01527054553122]
  We introduce networked communication to the mean-field game framework.<n>We prove that our architecture has sample guarantees bounded between those of the centralised- and independent-learning cases.<n>We show that our networked approach has significant advantages over both alternatives in terms of robustness to update failures and to changes in population size.
  arXiv  Detail & Related papers  (2023-06-05T10:45:39Z)
• Towards Robust Metrics for Concept Representation Evaluation [25.549961337814523]
  Concept learning models have been shown to be prone to encoding impurities in their representations. We propose novel metrics for evaluating the purity of concept representations in both approaches.
  arXiv  Detail & Related papers  (2023-01-25T00:40:19Z)
• Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI [60.142926537264714]
  We introduce the methodology of Faithfulness-through-Counterfactuals. It generates a counterfactual hypothesis based on the logical predicates expressed in the explanation. It then evaluates if the model's prediction on the counterfactual is consistent with that expressed logic.
  arXiv  Detail & Related papers  (2022-05-25T03:40:59Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Entity Concept-enhanced Few-shot Relation Extraction [35.10974511223129]
  Few-shot relation extraction (FSRE) is of great importance in long-tail distribution problem. Most existing FSRE algorithms fail to accurately classify the relations merely based on the information of the sentences together with the recognized entity pairs. We propose a novel entity-enhanced FEw-shot Relation Extraction scheme (ConceptFERE), which introduces the inherent concepts of entities to provide clues for relation prediction.
  arXiv  Detail & Related papers  (2021-06-04T10:36:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.