Why Does It Look There? Structured Explanations for Image Classification

• URL: http://arxiv.org/abs/2603.10234v1
• Date: Tue, 10 Mar 2026 21:15:14 GMT
• Title: Why Does It Look There? Structured Explanations for Image Classification
• Abstract summary: Interpretability to Explainability (I2X) is a framework that builds structured explanations directly from unstructured interpretability.<n>We show I2X can be used to improve predictions across different model architectures and datasets.
• Score: 31.62439146845817
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning models achieve remarkable predictive performance, yet their black-box nature limits transparency and trustworthiness. Although numerous explainable artificial intelligence (XAI) methods have been proposed, they primarily provide saliency maps or concepts (i.e., unstructured interpretability). Existing approaches often rely on auxiliary models (\eg, GPT, CLIP) to describe model behavior, thereby compromising faithfulness to the original models. We propose Interpretability to Explainability (I2X), a framework that builds structured explanations directly from unstructured interpretability by quantifying progress at selected checkpoints during training using prototypes extracted from post-hoc XAI methods (e.g., GradCAM). I2X answers the question of "why does it look there" by providing a structured view of both intra- and inter-class decision making during training. Experiments on MNIST and CIFAR10 demonstrate effectiveness of I2X to reveal prototype-based inference process of various image classification models. Moreover, we demonstrate that I2X can be used to improve predictions across different model architectures and datasets: we can identify uncertain prototypes recognized by I2X and then use targeted perturbation of samples that allows fine-tuning to ultimately improve accuracy. Thus, I2X not only faithfully explains model behavior but also provides a practical approach to guide optimization toward desired targets.

Related papers

• Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning [4.570003973862485]
  Multiple machine learning models can achieve near-equivalent predictive performance on the same task, yet provide divergent feature-based explanations.<n>This is called the Rashomon effect of (explainable) machine learning, and it raises the question of which explanations, if any, are trustworthy.<n>We propose a framework based on metamorphic testing that assesses explanation faithfulness without requiring ground-truth labels.
  arXiv  Detail & Related papers  (2026-06-04T11:57:26Z)
• VerLM: Explaining Face Verification Using Natural Language [50.56081916981731]
  We introduce an innovative Vision-Language Model (VLM) for Face Verification.<n>Our model is uniquely trained using two complementary explanation styles.<n>The proposed VLM integrates sophisticated feature extraction techniques with advanced reasoning capabilities, enabling clear articulation of its verification process.
  arXiv  Detail & Related papers  (2026-01-05T05:16:07Z)
• Enhancing Interpretability for Vision Models via Shapley Value Optimization [10.809438356590988]
  Self-explaining neural networks sacrifice performance and compatibility due to their specialized architectural designs.<n>We propose a novel self-explaining framework that integrates Shapley value estimation as an auxiliary task during training.<n>Our method achieves state-of-the-art interpretability.
  arXiv  Detail & Related papers  (2025-12-16T12:33:04Z)
• EPIC: Explanation of Pretrained Image Classification Networks via Prototype [6.619564359642066]
  EPIC (Explanation of Pretrained Image Classification) is a novel approach that bridges the gap between ante-hoc and post-hoc explanations.<n>It delivers intuitive, prototype-based explanations inspired by ante-hoc techniques.<n>We evaluate EPIC on benchmark datasets commonly used in prototype-based explanations.
  arXiv  Detail & Related papers  (2025-05-19T09:32:20Z)
• Harnessing Diffusion Models for Visual Perception with Meta Prompts [68.78938846041767]
  We propose a simple yet effective scheme to harness a diffusion model for visual perception tasks. We introduce learnable embeddings (meta prompts) to the pre-trained diffusion models to extract proper features for perception. Our approach achieves new performance records in depth estimation tasks on NYU depth V2 and KITTI, and in semantic segmentation task on CityScapes.
  arXiv  Detail & Related papers  (2023-12-22T14:40:55Z)
• Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models [9.945345991490624]
  We present Learning to Explain (LTX), a model-agnostic framework for providing post-hoc explanations for vision models. The LTX framework introduces an "explainer" model that generates explanation maps, highlighting the crucial regions that justify the predictions made by the model being explained. We demonstrate that LTX significantly outperforms the current state-of-the-art in explainability across various metrics.
  arXiv  Detail & Related papers  (2023-10-25T12:18:00Z)
• Flow Factorized Representation Learning [109.51947536586677]
  We introduce a generative model which specifies a distinct set of latent probability paths that define different input transformations. We show that our model achieves higher likelihoods on standard representation learning benchmarks while simultaneously being closer to approximately equivariant models.
  arXiv  Detail & Related papers  (2023-09-22T20:15:37Z)
• COSE: A Consistency-Sensitivity Metric for Saliency on Image Classification [21.3855970055692]
  We present a set of metrics that utilize vision priors to assess the performance of saliency methods on image classification tasks. We show that although saliency methods are thought to be architecture-independent, most methods could better explain transformer-based models over convolutional-based models.
  arXiv  Detail & Related papers  (2023-09-20T01:06:44Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• ContraFeat: Contrasting Deep Features for Semantic Discovery [102.4163768995288]
  StyleGAN has shown strong potential for disentangled semantic control. Existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results. We propose a model that automates this process and achieves state-of-the-art semantic discovery performance.
  arXiv  Detail & Related papers  (2022-12-14T15:22:13Z)
• Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classification [6.940242990198]
  Greybox XAI is a framework that composes a DNN and a transparent model thanks to the use of a symbolic Knowledge Base (KB) We address the problem of the lack of universal criteria for XAI by formalizing what an explanation is. We show how this new architecture is accurate and explainable in several datasets.
  arXiv  Detail & Related papers  (2022-09-26T08:55:31Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement [75.00655434905417]
  Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models. This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models. We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
  arXiv  Detail & Related papers  (2022-03-15T15:44:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.