Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models

• URL: http://arxiv.org/abs/2310.16584v1
• Date: Wed, 25 Oct 2023 12:18:00 GMT
• Title: Learning to Explain: A Model-Agnostic Framework for Explaining Black Box Models
• Abstract summary: We present Learning to Explain (LTX), a model-agnostic framework for providing post-hoc explanations for vision models. The LTX framework introduces an "explainer" model that generates explanation maps, highlighting the crucial regions that justify the predictions made by the model being explained. We demonstrate that LTX significantly outperforms the current state-of-the-art in explainability across various metrics.
• Score: 9.945345991490624
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Learning to Explain (LTX), a model-agnostic framework designed for providing post-hoc explanations for vision models. The LTX framework introduces an "explainer" model that generates explanation maps, highlighting the crucial regions that justify the predictions made by the model being explained. To train the explainer, we employ a two-stage process consisting of initial pretraining followed by per-instance finetuning. During both stages of training, we utilize a unique configuration where we compare the explained model's prediction for a masked input with its original prediction for the unmasked input. This approach enables the use of a novel counterfactual objective, which aims to anticipate the model's output using masked versions of the input image. Importantly, the LTX framework is not restricted to a specific model architecture and can provide explanations for both Transformer-based and convolutional models. Through our evaluations, we demonstrate that LTX significantly outperforms the current state-of-the-art in explainability across various metrics.

Related papers

• LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence [54.21298601916397]
  LimiX-2 is developed through model and data scaling guided by our previously established scaling laws.<n>LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling ( CCMM)<n> CCMM shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling.
  arXiv  Detail & Related papers  (2026-09-15T17:30:02Z)
• Why Does It Look There? Structured Explanations for Image Classification [31.62439146845817]
  Interpretability to Explainability (I2X) is a framework that builds structured explanations directly from unstructured interpretability.<n>We show I2X can be used to improve predictions across different model architectures and datasets.
  arXiv  Detail & Related papers  (2026-03-10T21:15:14Z)
• On the Origins of Linear Representations in Large Language Models [51.88404605700344]
  We introduce a simple latent variable model to formalize the concept dynamics of the next token prediction. Experiments show that linear representations emerge when learning from data matching the latent variable model. We additionally confirm some predictions of the theory using the LLaMA-2 large language model.
  arXiv  Detail & Related papers  (2024-03-06T17:17:36Z)
• Interpreting and Controlling Vision Foundation Models via Text Explanations [45.30541722925515]
  We present a framework for interpreting vision transformer's latent tokens with natural language. Our approach enables understanding of model visual reasoning procedure without needing additional model training or data collection.
  arXiv  Detail & Related papers  (2023-10-16T17:12:06Z)
• Evaluating and Explaining Large Language Models for Code Using Syntactic Structures [74.93762031957883]
  This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes. We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
  arXiv  Detail & Related papers  (2023-08-07T18:50:57Z)
• BELLA: Black box model Explanations by Local Linear Approximations [10.71090921516854]
  BELLA is a deterministic model-agnostic post-hoc approach for explaining the individual predictions of regression black-box models.<n>BELLA maximizes the size of the neighborhood to which the linear model applies so that the explanations are accurate, simple, general, and robust.
  arXiv  Detail & Related papers  (2023-05-18T21:22:23Z)
• Learning with Explanation Constraints [91.23736536228485]
  We provide a learning theoretic framework to analyze how explanations can improve the learning of our models. We demonstrate the benefits of our approach over a large array of synthetic and real-world experiments.
  arXiv  Detail & Related papers  (2023-03-25T15:06:47Z)
• Local Interpretable Model Agnostic Shap Explanations for machine learning models [0.0]
  We propose a methodology that we define as Local Interpretable Model Agnostic Shap Explanations (LIMASE) This proposed technique uses Shapley values under the LIME paradigm to achieve the following (a) explain prediction of any model by using a locally faithful and interpretable decision tree model on which the Tree Explainer is used to calculate the shapley values and give visually interpretable explanations.
  arXiv  Detail & Related papers  (2022-10-10T10:07:27Z)
• CARE: Coherent Actionable Recourse based on Sound Counterfactual Explanations [0.0]
  This paper introduces CARE, a modular explanation framework that addresses the model- and user-level desiderata. As a model-agnostic approach, CARE generates multiple, diverse explanations for any black-box model.
  arXiv  Detail & Related papers  (2021-08-18T15:26:59Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Demystifying Code Summarization Models [5.608277537412537]
  We evaluate four prominent code summarization models: extreme summarizer, code2vec, code2seq, and sequence GNN. Results show that all models base their predictions on syntactic and lexical properties with little to none semantic implication. We present a novel approach to explaining the predictions of code summarization models through the lens of training data.
  arXiv  Detail & Related papers  (2021-02-09T03:17:46Z)
• Deducing neighborhoods of classes from a fitted model [68.8204255655161]
  In this article a new kind of interpretable machine learning method is presented. It can help to understand the partitioning of the feature space into predicted classes in a classification model using quantile shifts. Basically, real data points (or specific points of interest) are used and the changes of the prediction after slightly raising or decreasing specific features are observed.
  arXiv  Detail & Related papers  (2020-09-11T16:35:53Z)
• Explanation-Guided Training for Cross-Domain Few-Shot Classification [96.12873073444091]
  Cross-domain few-shot classification task (CD-FSC) combines few-shot classification with the requirement to generalize across domains represented by datasets. We introduce a novel training approach for existing FSC models. We show that explanation-guided training effectively improves the model generalization.
  arXiv  Detail & Related papers  (2020-07-17T07:28:08Z)
• Explainable Deep Modeling of Tabular Data using TableGraphNet [1.376408511310322]
  We propose a new architecture that produces explainable predictions in the form of additive feature attributions. We show that our explainable model attains the same level of performance as black box models.
  arXiv  Detail & Related papers  (2020-02-12T20:02:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.