SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability

• URL: http://arxiv.org/abs/2208.09418v4
• Date: Mon, 31 Jul 2023 16:28:13 GMT
• Title: SAFARI: Versatile and Efficient Evaluations for Robustness of Interpretability
• Authors: Wei Huang, Xingyu Zhao, Gaojie Jin, Xiaowei Huang
• Abstract summary: Interpretability of Deep Learning (DL) is a barrier to trustworthy AI. It is vital to assess how robust DL interpretability is, given an XAI method.
• Score: 11.230696151134367
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Interpretability of Deep Learning (DL) is a barrier to trustworthy AI. Despite great efforts made by the Explainable AI (XAI) community, explanations lack robustness -- indistinguishable input perturbations may lead to different XAI results. Thus, it is vital to assess how robust DL interpretability is, given an XAI method. In this paper, we identify several challenges that the state-of-the-art is unable to cope with collectively: i) existing metrics are not comprehensive; ii) XAI techniques are highly heterogeneous; iii) misinterpretations are normally rare events. To tackle these challenges, we introduce two black-box evaluation methods, concerning the worst-case interpretation discrepancy and a probabilistic notion of how robust in general, respectively. Genetic Algorithm (GA) with bespoke fitness function is used to solve constrained optimisation for efficient worst-case evaluation. Subset Simulation (SS), dedicated to estimate rare event probabilities, is used for evaluating overall robustness. Experiments show that the accuracy, sensitivity, and efficiency of our methods outperform the state-of-the-arts. Finally, we demonstrate two applications of our methods: ranking robust XAI methods and selecting training schemes to improve both classification and interpretation robustness.

Related papers

• F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI [15.314388210699443]
  Fine-tuned Fidelity F-Fidelity is a robust evaluation framework for XAI. We show that F-Fidelity significantly improves upon prior evaluation metrics in recovering the ground-truth ranking of explainers. We also show that given a faithful explainer, F-Fidelity metric can be used to compute the sparsity of influential input components.
  arXiv  Detail & Related papers  (2024-10-03T20:23:06Z)
• Revisiting the robustness of post-hoc interpretability methods [1.5020330976600738]
  Post-hoc interpretability methods play a critical role in explainable artificial intelligence (XAI) Different post-hoc interpretability methods often provide different results, casting doubts on their accuracy. We propose an approach and two new metrics to provide a fine-grained assessment of post-hoc interpretability methods.
  arXiv  Detail & Related papers  (2024-07-29T03:55:52Z)
• Can you trust your explanations? A robustness test for feature attribution methods [42.36530107262305]
  The field of Explainable AI (XAI) has seen a rapid growth but the usage of its techniques has at times led to unexpected results. We will show how leveraging manifold hypothesis and ensemble approaches can be beneficial to an in-depth analysis of the robustness.
  arXiv  Detail & Related papers  (2024-06-20T14:17:57Z)
• ProTIP: Probabilistic Robustness Verification on Text-to-Image Diffusion Models against Stochastic Perturbation [18.103478658038846]
  Text-to-Image (T2I) Diffusion Models (DMs) have shown impressive abilities in generating high-quality images based on simple text descriptions. As is common with many Deep Learning (DL) models, DMs are subject to a lack of robustness. We introduce a probabilistic notion of T2I DMs' robustness; and then establish an efficient framework, ProTIP, to evaluate it with statistical guarantees.
  arXiv  Detail & Related papers  (2024-02-23T16:48:56Z)
• Doubly Robust Instance-Reweighted Adversarial Training [107.40683655362285]
  We propose a novel doubly-robust instance reweighted adversarial framework. Our importance weights are obtained by optimizing the KL-divergence regularized loss function. Our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance.
  arXiv  Detail & Related papers  (2023-08-01T06:16:18Z)
• An Experimental Investigation into the Evaluation of Explainability Methods [60.54170260771932]
  This work compares 14 different metrics when applied to nine state-of-the-art XAI methods and three dummy methods (e.g., random saliency maps) used as references. Experimental results show which of these metrics produces highly correlated results, indicating potential redundancy.
  arXiv  Detail & Related papers  (2023-05-25T08:07:07Z)
• The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus [10.135749005469686]
  One of the unsolved challenges in the field of Explainable AI (XAI) is determining how to most reliably estimate the quality of an explanation method. We address this issue through a meta-evaluation of different quality estimators in XAI. Our novel framework, MetaQuantus, analyses two complementary performance characteristics of a quality estimator.
  arXiv  Detail & Related papers  (2023-02-14T18:59:02Z)
• Probabilistically Robust Learning: Balancing Average- and Worst-case Performance [105.87195436925722]
  We propose a framework called robustness probabilistic that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case. From a theoretical point of view, this framework overcomes the trade-offs between the performance and the sample-complexity of worst-case and average-case learning.
  arXiv  Detail & Related papers  (2022-02-02T17:01:38Z)
• Adversarial Robustness with Semi-Infinite Constrained Learning [177.42714838799924]
  Deep learning to inputs perturbations has raised serious questions about its use in safety-critical domains. We propose a hybrid Langevin Monte Carlo training approach to mitigate this issue. We show that our approach can mitigate the trade-off between state-of-the-art performance and robust robustness.
  arXiv  Detail & Related papers  (2021-10-29T13:30:42Z)
• Probabilistic robust linear quadratic regulators with Gaussian processes [73.0364959221845]
  Probabilistic models such as Gaussian processes (GPs) are powerful tools to learn unknown dynamical systems from data for subsequent use in control design. We present a novel controller synthesis for linearized GP dynamics that yields robust controllers with respect to a probabilistic stability margin.
  arXiv  Detail & Related papers  (2021-05-17T08:36:18Z)
• Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
  Learning from implicit feedback is challenging because of the difficult nature of the one-class problem. Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem. We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
  arXiv  Detail & Related papers  (2021-05-11T03:38:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.