F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI

• URL: http://arxiv.org/abs/2410.02970v2
• Date: Thu, 06 Mar 2025 20:06:16 GMT
• Title: F-Fidelity: A Robust Framework for Faithfulness Evaluation of Explainable AI
• Authors: Xu Zheng, Farhad Shirani, Zhuomin Chen, Chaohao Lin, Wei Cheng, Wenbo Guo, Dongsheng Luo,
• Abstract summary: XAI techniques can extract meaningful insights from deep learning models.<n>How to properly evaluate them remains an open problem.<n>We propose Fine-tuned Fidelity (F-Fidelity) as a robust evaluation framework for XAI.
• Score: 15.314388210699443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent research has developed a number of eXplainable AI (XAI) techniques, such as gradient-based approaches, input perturbation-base methods, and black-box explanation methods. While these XAI techniques can extract meaningful insights from deep learning models, how to properly evaluate them remains an open problem. The most widely used approach is to perturb or even remove what the XAI method considers to be the most important features in an input and observe the changes in the output prediction. This approach, although straightforward, suffers the Out-of-Distribution (OOD) problem as the perturbed samples may no longer follow the original data distribution. A recent method RemOve And Retrain (ROAR) solves the OOD issue by retraining the model with perturbed samples guided by explanations. However, using the model retrained based on XAI methods to evaluate these explainers may cause information leakage and thus lead to unfair comparisons. We propose Fine-tuned Fidelity (F-Fidelity), a robust evaluation framework for XAI, which utilizes i) an explanation-agnostic fine-tuning strategy, thus mitigating the information leakage issue, and ii) a random masking operation that ensures that the removal step does not generate an OOD input. We also design controlled experiments with state-of-the-art (SOTA) explainers and their degraded version to verify the correctness of our framework. We conduct experiments on multiple data modalities, such as images, time series, and natural language. The results demonstrate that F-Fidelity significantly improves upon prior evaluation metrics in recovering the ground-truth ranking of the explainers. Furthermore, we show both theoretically and empirically that, given a faithful explainer, F-Fidelity metric can be used to compute the sparsity of influential input components, i.e., to extract the true explanation size.

Related papers

• PEEL the Layers and Find Yourself: Revisiting Inference-time Data Leakage for Residual Neural Networks [64.90981115460937]
  This paper explores inference-time data leakage risks of deep neural networks (NNs) We propose a novel backward feature inversion method, textbfPEEL, which can effectively recover block-wise input features from the intermediate output of residual NNs. Our results show that PEEL outperforms the state-of-the-art recovery methods by an order of magnitude when evaluated by mean squared error (MSE)
  arXiv  Detail & Related papers  (2025-04-08T20:11:05Z)
• Effort: Efficient Orthogonal Modeling for Generalizable AI-Generated Image Detection [66.16595174895802]
  Existing AI-generated image (AIGI) detection methods often suffer from limited generalization performance. In this paper, we identify a crucial yet previously overlooked asymmetry phenomenon in AIGI detection.
  arXiv  Detail & Related papers  (2024-11-23T19:10:32Z)
• Attribute-to-Delete: Machine Unlearning via Datamodel Matching [65.13151619119782]
  Machine unlearning -- efficiently removing a small "forget set" training data on a pre-divertrained machine learning model -- has recently attracted interest. Recent research shows that machine unlearning techniques do not hold up in such a challenging setting.
  arXiv  Detail & Related papers  (2024-10-30T17:20:10Z)
• Why Fine-Tuning Struggles with Forgetting in Machine Unlearning? Theoretical Insights and a Remedial Approach [19.307968983872588]
  Fine-tuning (FT) methods have become one of the fundamental approaches for approximating unlearning. We present the first theoretical analysis of FT methods for machine unlearning within a linear regression framework. We propose a theoretical approach to mitigate the retention of forgetting data in the pretrained model.
  arXiv  Detail & Related papers  (2024-10-04T18:01:52Z)
• Robustness of Explainable Artificial Intelligence in Industrial Process Modelling [43.388607981317016]
  We evaluate current XAI methods by scoring them based on ground truth simulations and sensitivity analysis. We show the differences between XAI methods in their ability to correctly predict the true sensitivity of the modeled industrial process.
  arXiv  Detail & Related papers  (2024-07-12T09:46:26Z)
• CHILLI: A data context-aware perturbation method for XAI [3.587367153279351]
  The trustworthiness of Machine Learning (ML) models can be difficult to assess, but is critical in high-risk or ethically sensitive applications. We propose a novel framework, CHILLI, for incorporating data context into XAI by generating contextually aware perturbations. This is shown to improve both the soundness and accuracy of the explanations.
  arXiv  Detail & Related papers  (2024-07-10T10:18:07Z)
• Explainability of Machine Learning Models under Missing Data [2.880748930766428]
  Missing data is a prevalent issue that can significantly impair model performance and interpretability. This paper briefly summarizes the development of the field of missing data and investigates the effects of various imputation methods on the calculation of Shapley values.
  arXiv  Detail & Related papers  (2024-06-29T11:31:09Z)
• Assessing Fidelity in XAI post-hoc techniques: A Comparative Study with Ground Truth Explanations Datasets [0.0]
  XAI methods based on the backpropagation of output information to input yield higher accuracy and reliability. Backpropagation method tends to generate more noisy saliency maps. Findings have significant implications for the advancement of XAI methods.
  arXiv  Detail & Related papers  (2023-11-03T14:57:24Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Precise Benchmarking of Explainable AI Attribution Methods [0.0]
  We propose a novel evaluation approach for benchmarking state-of-the-art XAI attribution methods. Our proposal consists of a synthetic classification model accompanied by its derived ground truth explanations. Our experimental results provide novel insights into the performance of Guided-Backprop and Smoothgrad XAI methods.
  arXiv  Detail & Related papers  (2023-08-06T17:03:32Z)
• Interpretable pipelines with evolutionarily optimized modules for RL tasks with visual inputs [5.254093731341154]
  We propose end-to-end pipelines composed of multiple interpretable models co-optimized by means of evolutionary algorithms. We test our approach in reinforcement learning environments from the Atari benchmark.
  arXiv  Detail & Related papers  (2022-02-10T10:33:44Z)
• Agree to Disagree: Diversity through Disagreement for Better Transferability [54.308327969778155]
  We propose D-BAT (Diversity-By-disAgreement Training), which enforces agreement among the models on the training data. We show how D-BAT naturally emerges from the notion of generalized discrepancy.
  arXiv  Detail & Related papers  (2022-02-09T12:03:02Z)
• Leveraging Unlabeled Data to Predict Out-of-Distribution Performance [63.740181251997306]
  Real-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions. In this work, we investigate methods for predicting the target domain accuracy using only labeled source data and unlabeled target data. We propose Average Thresholded Confidence (ATC), a practical method that learns a threshold on the model's confidence, predicting accuracy as the fraction of unlabeled examples.
  arXiv  Detail & Related papers  (2022-01-11T23:01:12Z)
• Imputation-Free Learning from Incomplete Observations [73.15386629370111]
  We introduce the importance of guided gradient descent (IGSGD) method to train inference from inputs containing missing values without imputation. We employ reinforcement learning (RL) to adjust the gradients used to train the models via back-propagation. Our imputation-free predictions outperform the traditional two-step imputation-based predictions using state-of-the-art imputation methods.
  arXiv  Detail & Related papers  (2021-07-05T12:44:39Z)
• A new interpretable unsupervised anomaly detection method based on residual explanation [47.187609203210705]
  We present RXP, a new interpretability method to deal with the limitations for AE-based AD in large-scale systems. It stands out for its implementation simplicity, low computational cost and deterministic behavior. In an experiment using data from a real heavy-haul railway line, the proposed method achieved superior performance compared to SHAP.
  arXiv  Detail & Related papers  (2021-03-14T15:35:45Z)
• Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? [97.77183117452235]
  We carry out human subject tests to isolate the effect of algorithmic explanations on model interpretability. Clear evidence of method effectiveness is found in very few cases. Our results provide the first reliable and comprehensive estimates of how explanations influence simulatability.
  arXiv  Detail & Related papers  (2020-05-04T20:35:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.