Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoff

• URL: http://arxiv.org/abs/2107.05693v1
• Date: Mon, 12 Jul 2021 19:07:24 GMT
• Title: Quantifying Explainability in NLP and Analyzing Algorithms for Performance-Explainability Tradeoff
• Authors: Mitchell Naylor, Christi French, Samantha Terker, Uday Kamath
• Abstract summary: We explore the current art of explainability and interpretability within a case study in clinical text classification. We demonstrate various visualization techniques for fully interpretable methods as well as model-agnostic post hoc attributions. We introduce a framework through which practitioners and researchers can assess the frontier between a model's predictive performance and the quality of its available explanations.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: The healthcare domain is one of the most exciting application areas for machine learning, but a lack of model transparency contributes to a lag in adoption within the industry. In this work, we explore the current art of explainability and interpretability within a case study in clinical text classification, using a task of mortality prediction within MIMIC-III clinical notes. We demonstrate various visualization techniques for fully interpretable methods as well as model-agnostic post hoc attributions, and we provide a generalized method for evaluating the quality of explanations using infidelity and local Lipschitz across model types from logistic regression to BERT variants. With these metrics, we introduce a framework through which practitioners and researchers can assess the frontier between a model's predictive performance and the quality of its available explanations. We make our code available to encourage continued refinement of these methods.

Related papers

• Methodological Explainability Evaluation of an Interpretable Deep Learning Model for Post-Hepatectomy Liver Failure Prediction Incorporating Counterfactual Explanations and Layerwise Relevance Propagation: A Prospective In Silico Trial [13.171582596404313]
  We developed a variational autoencoder-multilayer perceptron (VAE-MLP) model for preoperative PHLF prediction. This model integrated counterfactuals and layerwise relevance propagation (LRP) to provide insights into its decision-making mechanism. Results from the three-track in silico clinical trial showed that clinicians' prediction accuracy and confidence increased when AI explanations were provided.
  arXiv  Detail & Related papers  (2024-08-07T13:47:32Z)
• Prospector Heads: Generalized Feature Attribution for Large Models & Data [82.02696069543454]
  We introduce prospector heads, an efficient and interpretable alternative to explanation-based attribution methods. We demonstrate how prospector heads enable improved interpretation and discovery of class-specific patterns in input data.
  arXiv  Detail & Related papers  (2024-02-18T23:01:28Z)
• MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning [48.97640824497327]
  We propose a novel framework leveraging domain-specific medical knowledge as guiding signals to integrate language information into the visual domain through image-text contrastive learning. Our model includes global contrastive learning with our designed divergence encoder, local token-knowledge-patch alignment contrastive learning, and knowledge-guided category-level contrastive learning with expert knowledge. Notably, MLIP surpasses state-of-the-art methods even with limited annotated data, highlighting the potential of multimodal pre-training in advancing medical representation learning.
  arXiv  Detail & Related papers  (2024-02-03T05:48:50Z)
• SHAMSUL: Systematic Holistic Analysis to investigate Medical Significance Utilizing Local interpretability methods in deep learning for chest radiography pathology prediction [1.0138723409205497]
  The study delves into the application of four well-established interpretability methods: Local Interpretable Model-agnostic Explanations (LIME), Shapley Additive exPlanations (SHAP), Gradient-weighted Class Activation Mapping (Grad-CAM) and Layer-wise Relevance Propagation (LRP) Our analysis encompasses both single-label and multi-label predictions, providing a comprehensive and unbiased assessment through quantitative and qualitative investigations, which are compared against human expert annotation.
  arXiv  Detail & Related papers  (2023-07-16T11:10:35Z)
• Investigating Poor Performance Regions of Black Boxes: LIME-based Exploration in Sepsis Detection [0.5872014229110214]
  This paper proposes leveraging Local Interpretable Model-Agnostic Explanations (LIME) to provide interpretable descriptions of black box classification models in sepsis detection. By analyzing misclassified instances, significant features contributing to suboptimal performance are identified.
  arXiv  Detail & Related papers  (2023-06-21T18:36:15Z)
• Learning disentangled representations for explainable chest X-ray classification using Dirichlet VAEs [68.73427163074015]
  This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. The predictive capacity of multi-modal latent representations learned by DirVAE models is investigated through implementation of an auxiliary multi-label classification task.
  arXiv  Detail & Related papers  (2023-02-06T18:10:08Z)
• MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation [132.77005365032468]
  We propose a novel framework of Model-Agnostic Counterfactual Explanation (MACE) In our MACE approach, we propose a novel RL-based method for finding good counterfactual examples and a gradient-less descent method for improving proximity. Experiments on public datasets validate the effectiveness with better validity, sparsity and proximity.
  arXiv  Detail & Related papers  (2022-05-31T04:57:06Z)
• Explainability in Process Outcome Prediction: Guidelines to Obtain Interpretable and Faithful Models [77.34726150561087]
  We define explainability through the interpretability of the explanations and the faithfulness of the explainability model in the field of process outcome prediction. This paper contributes a set of guidelines named X-MOP which allows selecting the appropriate model based on the event log specifications.
  arXiv  Detail & Related papers  (2022-03-30T05:59:50Z)
• TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Models [0.0]
  Clinicians are often sceptical about applying automatic image processing approaches, especially deep learning based methods, in practice. This paper presents approaches that help to interpret and explain the results of deep learning algorithms by depicting the anatomical areas which influence the decision of the algorithm most. Research presents a unified framework, TorchEsegeta, for applying various interpretability and explainability techniques for deep learning models.
  arXiv  Detail & Related papers  (2021-10-16T01:00:15Z)
• A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
  We develop a list of diagnostic properties for evaluating existing explainability techniques. We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
  arXiv  Detail & Related papers  (2020-09-25T12:01:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.