Towards Better Understanding Attribution Methods

• URL: http://arxiv.org/abs/2205.10435v1
• Date: Fri, 20 May 2022 20:50:17 GMT
• Title: Towards Better Understanding Attribution Methods
• Authors: Sukrut Rao, Moritz B\"ohle, Bernt Schiele
• Abstract summary: Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
• Score: 77.1487219861185
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks are very successful on many vision tasks, but hard to interpret due to their black box nature. To overcome this, various post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. Evaluating such methods is challenging since no ground truth attributions exist. We thus propose three novel evaluation schemes to more reliably measure the faithfulness of those methods, to make comparisons between them more fair, and to make visual inspection more systematic. To address faithfulness, we propose a novel evaluation setting (DiFull) in which we carefully control which parts of the input can influence the output in order to distinguish possible from impossible attributions. To address fairness, we note that different methods are applied at different layers, which skews any comparison, and so evaluate all methods on the same layers (ML-Att) and discuss how this impacts their performance on quantitative metrics. For more systematic visualizations, we propose a scheme (AggAtt) to qualitatively evaluate the methods on complete datasets. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods. Finally, we propose a post-processing smoothing step that significantly improves the performance of some attribution methods, and discuss its applicability.

Related papers

• Toward Understanding the Disagreement Problem in Neural Network Feature Attribution [0.8057006406834466]
  neural networks have demonstrated their remarkable ability to discern intricate patterns and relationships from raw data. Understanding the inner workings of these black box models remains challenging, yet crucial for high-stake decisions. Our work addresses this confusion by investigating the explanations' fundamental and distributional behavior.
  arXiv  Detail & Related papers  (2024-04-17T12:45:59Z)
• A Large-Scale Empirical Study on Improving the Fairness of Image Classification Models [22.522156479335706]
  This paper conducts the first large-scale empirical study to compare the performance of existing state-of-the-art fairness improving techniques. Our findings reveal substantial variations in the performance of each method across different datasets and sensitive attributes. Different fairness evaluation metrics, due to their distinct focuses, yield significantly different assessment results.
  arXiv  Detail & Related papers  (2024-01-08T06:53:33Z)
• Towards Evaluating Transfer-based Attacks Systematically, Practically, and Fairly [79.07074710460012]
  adversarial vulnerability of deep neural networks (DNNs) has drawn great attention. An increasing number of transfer-based methods have been developed to fool black-box DNN models. We establish a transfer-based attack benchmark (TA-Bench) which implements 30+ methods.
  arXiv  Detail & Related papers  (2023-11-02T15:35:58Z)
• Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluation [64.0476282000118]
  Latest methods for visual counterfactual explanations (VCE) harness the power of deep generative models to synthesize new examples of high-dimensional images of impressive quality. It is currently difficult to compare the performance of these VCE methods as the evaluation procedures largely vary and often boil down to visual inspection of individual examples and small scale user studies. We propose a framework for systematic, quantitative evaluation of the VCE methods and a minimal set of metrics to be used.
  arXiv  Detail & Related papers  (2023-08-11T12:22:37Z)
• Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
  arXiv  Detail & Related papers  (2023-03-21T14:24:58Z)
• Time to Focus: A Comprehensive Benchmark Using Time Series Attribution Methods [4.9449660544238085]
  The paper focuses on time series analysis and benchmark several state-of-the-art attribution methods. The presented experiments involve gradient-based and perturbation-based attribution methods. The findings accentuate that choosing the best-suited attribution method is strongly correlated with the desired use case.
  arXiv  Detail & Related papers  (2022-02-08T10:06:13Z)
• FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding [89.92513889132825]
  We introduce an evaluation framework that improves previous evaluation procedures in three key aspects, i.e., test performance, dev-test correlation, and stability. We open-source our toolkit, FewNLU, that implements our evaluation framework along with a number of state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-09-27T00:57:30Z)
• Revisiting The Evaluation of Class Activation Mapping for Explainability: A Novel Metric and Experimental Analysis [54.94682858474711]
  Class Activation Mapping (CAM) approaches provide an effective visualization by taking weighted averages of the activation maps. We propose a novel set of metrics to quantify explanation maps, which show better effectiveness and simplify comparisons between approaches.
  arXiv  Detail & Related papers  (2021-04-20T21:34:24Z)
• On the Ambiguity of Rank-Based Evaluation of Entity Alignment or Link Prediction Methods [27.27230441498167]
  We take a closer look at the evaluation of two families of methods for enriching information from knowledge graphs: Link Prediction and Entity Alignment. In particular, we demonstrate that all existing scores can hardly be used to compare results across different datasets. We show that this leads to various problems in the interpretation of results, which may support misleading conclusions.
  arXiv  Detail & Related papers  (2020-02-17T12:26:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.