The Weighting Game: Evaluating Quality of Explainability Methods
- URL: http://arxiv.org/abs/2208.06175v1
- Date: Fri, 12 Aug 2022 08:50:21 GMT
- Title: The Weighting Game: Evaluating Quality of Explainability Methods
- Authors: Lassi Raatikainen and Esa Rahtu
- Abstract summary: We introduce the Weighting Game, which measures how much of a class-guided explanation is contained within the correct class' segmentation mask.
Secondly, we introduce a metric for explanation stability, using zooming/panning transformations to measure differences between saliency maps with similar contents.
Quantitative experiments are produced, using these new metrics, to evaluate the quality of explanations provided by commonly used CAM methods.
- Score: 14.632777952261716
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The objective of this paper is to assess the quality of explanation heatmaps
for image classification tasks. To assess the quality of explainability
methods, we approach the task through the lens of accuracy and stability.
In this work, we make the following contributions. Firstly, we introduce the
Weighting Game, which measures how much of a class-guided explanation is
contained within the correct class' segmentation mask. Secondly, we introduce a
metric for explanation stability, using zooming/panning transformations to
measure differences between saliency maps with similar contents.
Quantitative experiments are produced, using these new metrics, to evaluate
the quality of explanations provided by commonly used CAM methods. The quality
of explanations is also contrasted between different model architectures, with
findings highlighting the need to consider model architecture when choosing an
explainability method.
Related papers
- Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly.
Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness.
Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings.
This paper introduces an innovative multi-modal prompt-based methodology for IQA.
arXiv Detail & Related papers (2024-04-23T11:45:32Z) - Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development.
To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps.
These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
arXiv Detail & Related papers (2023-12-10T23:13:23Z) - On the stability, correctness and plausibility of visual explanation
methods based on feature importance [0.0]
We study the articulation between the stability, correctness and plausibility of explanations based on feature importance for image classifiers.
We show that the existing metrics for evaluating these properties do not always agree, raising the issue of what constitutes a good evaluation metric for explanations.
arXiv Detail & Related papers (2023-10-25T08:59:21Z) - COSE: A Consistency-Sensitivity Metric for Saliency on Image
Classification [21.3855970055692]
We present a set of metrics that utilize vision priors to assess the performance of saliency methods on image classification tasks.
We show that although saliency methods are thought to be architecture-independent, most methods could better explain transformer-based models over convolutional-based models.
arXiv Detail & Related papers (2023-09-20T01:06:44Z) - An Explainable Model-Agnostic Algorithm for CNN-based Biometrics
Verification [55.28171619580959]
This paper describes an adaptation of the Local Interpretable Model-Agnostic Explanations (LIME) AI method to operate under a biometric verification setting.
arXiv Detail & Related papers (2023-07-25T11:51:14Z) - Clarity: an improved gradient method for producing quality visual
counterfactual explanations [7.279730418361996]
Visual counterfactual explanations identify modifications to an image that would change the prediction of a classifier.
We propose techniques based on generative models (VAE) and a classifier ensemble directly trained in the latent space.
These improvements lead to a novel classification model, Clarity, which produces realistic counterfactual explanations over all images.
arXiv Detail & Related papers (2022-11-22T10:53:17Z) - REVEL Framework to measure Local Linear Explanations for black-box
models: Deep Learning Image Classification case of study [12.49538398746092]
We propose a procedure called REVEL to evaluate different aspects concerning the quality of explanations with a theoretically coherent development.
The experiments have been carried out on image four datasets as benchmark where we show REVEL's descriptive and analytical power.
arXiv Detail & Related papers (2022-11-11T12:15:36Z) - Revisiting The Evaluation of Class Activation Mapping for
Explainability: A Novel Metric and Experimental Analysis [54.94682858474711]
Class Activation Mapping (CAM) approaches provide an effective visualization by taking weighted averages of the activation maps.
We propose a novel set of metrics to quantify explanation maps, which show better effectiveness and simplify comparisons between approaches.
arXiv Detail & Related papers (2021-04-20T21:34:24Z) - Explainers in the Wild: Making Surrogate Explainers Robust to
Distortions through Perception [77.34726150561087]
We propose a methodology to evaluate the effect of distortions in explanations by embedding perceptual distances.
We generate explanations for images in the Imagenet-C dataset and demonstrate how using a perceptual distances in the surrogate explainer creates more coherent explanations for the distorted and reference images.
arXiv Detail & Related papers (2021-02-22T12:38:53Z) - Explainable Image Classification with Evidence Counterfactual [0.0]
We introduce SEDC as a model-agnostic instance-level explanation method for image classification.
For a given image, SEDC searches a small set of segments that, in case of removal, alters the classification.
We compare SEDC(-T) with popular feature importance methods such as LRP, LIME and SHAP, and we describe how the mentioned importance ranking issues are addressed.
arXiv Detail & Related papers (2020-04-16T08:02:48Z) - There and Back Again: Revisiting Backpropagation Saliency Methods [87.40330595283969]
Saliency methods seek to explain the predictions of a model by producing an importance map across each input sample.
A popular class of such methods is based on backpropagating a signal and analyzing the resulting gradient.
We propose a single framework under which several such methods can be unified.
arXiv Detail & Related papers (2020-04-06T17:58:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.