Developing a Fidelity Evaluation Approach for Interpretable Machine Learning

• URL: http://arxiv.org/abs/2106.08492v1
• Date: Wed, 16 Jun 2021 00:21:16 GMT
• Title: Developing a Fidelity Evaluation Approach for Interpretable Machine Learning
• Authors: Mythreyi Velmurugan and Chun Ouyang and Catarina Moreira and Renuka Sindhgatta
• Abstract summary: Explainable AI (XAI) methods are used to improve the interpretability of complex models. In particular, methods to evaluate the fidelity of the explanation to the underlying black box require further development. Our evaluations suggest that the internal mechanism of the underlying predictive model, the internal mechanism of the explainable method used and model and data complexity all affect explanation fidelity.
• Score: 2.2448567386846916
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Although modern machine learning and deep learning methods allow for complex and in-depth data analytics, the predictive models generated by these methods are often highly complex, and lack transparency. Explainable AI (XAI) methods are used to improve the interpretability of these complex models, and in doing so improve transparency. However, the inherent fitness of these explainable methods can be hard to evaluate. In particular, methods to evaluate the fidelity of the explanation to the underlying black box require further development, especially for tabular data. In this paper, we (a) propose a three phase approach to developing an evaluation method; (b) adapt an existing evaluation method primarily for image and text data to evaluate models trained on tabular data; and (c) evaluate two popular explainable methods using this evaluation method. Our evaluations suggest that the internal mechanism of the underlying predictive model, the internal mechanism of the explainable method used and model and data complexity all affect explanation fidelity. Given that explanation fidelity is so sensitive to context and tools and data used, we could not clearly identify any specific explainable method as being superior to another.

Related papers

• Beyond the Veil of Similarity: Quantifying Semantic Continuity in Explainable AI [1.628012064605754]
  We introduce a novel metric for measuring semantic continuity in Explainable AI methods and machine learning models. We conduct experiments to observe how incremental changes in input affect the explanations provided by different XAI methods.
  arXiv  Detail & Related papers  (2024-07-17T18:32:41Z)
• Exploring the Trade-off Between Model Performance and Explanation Plausibility of Text Classifiers Using Human Rationales [3.242050660144211]
  Saliency post-hoc explainability methods are important tools for understanding increasingly complex NLP models. We present a methodology for incorporating rationales, which are text annotations explaining human decisions, into text classification models.
  arXiv  Detail & Related papers  (2024-04-03T22:39:33Z)
• A Survey of Explainable Knowledge Tracing [14.472784840283099]
  This paper thoroughly analyzes the interpretability of KT algorithms. Current evaluation methods for explainable knowledge tracing are lacking. This paper offers some insights into evaluation methods from the perspective of educational stakeholders.
  arXiv  Detail & Related papers  (2024-03-12T03:17:59Z)
• Explainability for Machine Learning Models: From Data Adaptability to User Perception [0.8702432681310401]
  This thesis explores the generation of local explanations for already deployed machine learning models. It aims to identify optimal conditions for producing meaningful explanations considering both data and user requirements.
  arXiv  Detail & Related papers  (2024-02-16T18:44:37Z)
• Evaluating the Utility of Model Explanations for Model Development [54.23538543168767]
  We evaluate whether explanations can improve human decision-making in practical scenarios of machine learning model development. To our surprise, we did not find evidence of significant improvement on tasks when users were provided with any of the saliency maps. These findings suggest caution regarding the usefulness and potential for misunderstanding in saliency-based explanations.
  arXiv  Detail & Related papers  (2023-12-10T23:13:23Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Beyond Explaining: Opportunities and Challenges of XAI-Based Model Improvement [75.00655434905417]
  Explainable Artificial Intelligence (XAI) is an emerging research field bringing transparency to highly complex machine learning (ML) models. This paper offers a comprehensive overview over techniques that apply XAI practically for improving various properties of ML models. We show empirically through experiments on toy and realistic settings how explanations can help improve properties such as model generalization ability or reasoning.
  arXiv  Detail & Related papers  (2022-03-15T15:44:28Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Distilling Interpretable Models into Human-Readable Code [71.11328360614479]
  Human-readability is an important and desirable standard for machine-learned model interpretability. We propose to train interpretable models using conventional methods, and then distill them into concise, human-readable code. We describe a piecewise-linear curve-fitting algorithm that produces high-quality results efficiently and reliably across a broad range of use cases.
  arXiv  Detail & Related papers  (2021-01-21T01:46:36Z)
• Evaluating Explainable Methods for Predictive Process Analytics: A Functionally-Grounded Approach [2.2448567386846916]
  Predictive process analytics focuses on predicting the future states of running instances of a business process. Current explainable machine learning methods, such as LIME and SHAP, can be used to interpret black box models. We apply the proposed metrics to evaluate the performance of LIME and SHAP in interpreting process predictive models built on XGBoost.
  arXiv  Detail & Related papers  (2020-12-08T05:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.