Evaluating and Aggregating Feature-based Model Explanations

• URL: http://arxiv.org/abs/2005.00631v1
• Date: Fri, 1 May 2020 21:56:36 GMT
• Title: Evaluating and Aggregating Feature-based Model Explanations
• Authors: Umang Bhatt, Adrian Weller, and Jos\'e M. F. Moura
• Abstract summary: A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity.
• Score: 27.677158604772238
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.

Related papers

• Selective Explanations [14.312717332216073]
  A machine learning model is trained to predict feature attribution scores with only one inference. Despite their efficiency, amortized explainers can produce inaccurate predictions and misleading explanations. We propose selective explanations, a novel feature attribution method that detects when amortized explainers generate low-quality explanations.
  arXiv  Detail & Related papers  (2024-05-29T23:08:31Z)
• Succinct Interaction-Aware Explanations [33.25637826682827]
  SHAP is a popular approach to explain black-box models by revealing the importance of individual features. NSHAP, on the other hand, reports the additive importance for all subsets of features. We propose to combine the best of these two worlds, by partitioning the features into parts that significantly interact.
  arXiv  Detail & Related papers  (2024-02-08T11:04:11Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• FIND: A Function Description Benchmark for Evaluating Interpretability Methods [86.80718559904854]
  This paper introduces FIND (Function INterpretation and Description), a benchmark suite for evaluating automated interpretability methods. FIND contains functions that resemble components of trained neural networks, and accompanying descriptions of the kind we seek to generate. We evaluate methods that use pretrained language models to produce descriptions of function behavior in natural language and code.
  arXiv  Detail & Related papers  (2023-09-07T17:47:26Z)
• Explaining the Model and Feature Dependencies by Decomposition of the Shapley Value [3.0655581300025996]
  Shapley values have become one of the go-to methods to explain complex models to end-users. One downside is that they always require outputs of the model when some features are missing. This however introduces a non-trivial choice: do we condition on the unknown features or not? We propose a new algorithmic approach to combine both explanations, removing the burden of choice and enhancing the explanatory power of Shapley values.
  arXiv  Detail & Related papers  (2023-06-19T12:20:23Z)
• ExaRanker: Explanation-Augmented Neural Ranker [67.4894325619275]
  In this work, we show that neural rankers also benefit from explanations. We use LLMs such as GPT-3.5 to augment retrieval datasets with explanations. Our model, dubbed ExaRanker, finetuned on a few thousand examples with synthetic explanations performs on par with models finetuned on 3x more examples without explanations.
  arXiv  Detail & Related papers  (2023-01-25T11:03:04Z)
• Diagnostics-Guided Explanation Generation [32.97930902104502]
  Explanations shed light on a machine learning model's rationales and can aid in identifying deficiencies in its reasoning process. We show how to optimise for several diagnostic properties when training a model to generate sentence-level explanations.
  arXiv  Detail & Related papers  (2021-09-08T16:27:52Z)
• Search Methods for Sufficient, Socially-Aligned Feature Importance Explanations with In-Distribution Counterfactuals [72.00815192668193]
  Feature importance (FI) estimates are a popular form of explanation, and they are commonly created and evaluated by computing the change in model confidence caused by removing certain input features at test time. We study several under-explored dimensions of FI-based explanations, providing conceptual and empirical improvements for this form of explanation.
  arXiv  Detail & Related papers  (2021-06-01T20:36:48Z)
• Towards Unifying Feature Attribution and Counterfactual Explanations: Different Means to the Same End [17.226134854746267]
  We present a method to generate feature attribution explanations from a set of counterfactual examples. We show how counterfactual examples can be used to evaluate the goodness of an attribution-based explanation in terms of its necessity and sufficiency.
  arXiv  Detail & Related papers  (2020-11-10T05:41:43Z)
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets [61.66584140190247]
  We show that feature-based explanations pose problems even for explaining trivial models. We show that two popular classes of explainers, Shapley explainers and minimal sufficient subsets explainers, target fundamentally different types of ground-truth explanations.
  arXiv  Detail & Related papers  (2020-09-23T09:45:23Z)
• Evaluations and Methods for Explanation through Robustness Analysis [117.7235152610957]
  We establish a novel set of evaluation criteria for such feature based explanations by analysis. We obtain new explanations that are loosely necessary and sufficient for a prediction. We extend the explanation to extract the set of features that would move the current prediction to a target class.
  arXiv  Detail & Related papers  (2020-05-31T05:52:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.