Towards a perturbation-based explanation for medical AI as differentiable programs

• URL: http://arxiv.org/abs/2502.14001v1
• Date: Wed, 19 Feb 2025 07:56:23 GMT
• Title: Towards a perturbation-based explanation for medical AI as differentiable programs
• Authors: Takeshi Abe, Yoshiyuki Asai,
• Abstract summary: In medicine and healthcare, there is a particular demand for sufficient and objective explainability of the outcome generated by AI models. This work examines a numerical availability of the Jacobian matrix of deep learning models that measures how stably a model responses against small perturbations added to the input. This is a first step towards a perturbation-based explanation, which will assist medical practitioners in understanding and interpreting the response of the AI model in its clinical application.
• Score: 0.0
• License:
• Abstract: Recent advancement in machine learning algorithms reaches a point where medical devices can be equipped with artificial intelligence (AI) models for diagnostic support and routine automation in clinical settings. In medicine and healthcare, there is a particular demand for sufficient and objective explainability of the outcome generated by AI models. However, AI models are generally considered as black boxes due to their complexity, and the computational process leading to their response is often opaque. Although several methods have been proposed to explain the behavior of models by evaluating the importance of each feature in discrimination and prediction, they may suffer from biases and opacities arising from the scale and sampling protocol of the dataset used for training or testing. To overcome the shortcomings of existing methods, we explore an alternative approach to provide an objective explanation of AI models that can be defined independently of the learning process and does not require additional data. As a preliminary study for this direction of research, this work examines a numerical availability of the Jacobian matrix of deep learning models that measures how stably a model responses against small perturbations added to the input. The indicator, if available, are calculated from a trained AI model for a given target input. This is a first step towards a perturbation-based explanation, which will assist medical practitioners in understanding and interpreting the response of the AI model in its clinical application.

Related papers

• Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling [10.014130930114172]
  We introduce a conditional generative AI model designed for virtual clinical trials (VCTs) of radiology AI. By learning the joint distribution of images and anatomical structures, our model enables precise replication of real-world patient populations. We demonstrate meaningful evaluation of radiology AI models through VCTs powered by our synthetic CT study populations.
  arXiv  Detail & Related papers  (2025-02-13T15:53:52Z)
• Selecting Interpretability Techniques for Healthcare Machine Learning models [69.65384453064829]
  In healthcare there is a pursuit for employing interpretable algorithms to assist healthcare professionals in several decision scenarios. We overview a selection of eight algorithms, both post-hoc and model-based, that can be used for such purposes.
  arXiv  Detail & Related papers  (2024-06-14T17:49:04Z)
• QXAI: Explainable AI Framework for Quantitative Analysis in Patient Monitoring Systems [9.29069202652354]
  An Explainable AI for Quantitative analysis (QXAI) framework is proposed with post-hoc model explainability and intrinsic explainability for regression and classification tasks. We adopted the artificial neural networks (ANN) and attention-based Bidirectional LSTM (BiLSTM) models for the prediction of heart rate and classification of physical activities based on sensor data.
  arXiv  Detail & Related papers  (2023-09-19T03:50:30Z)
• GENIE-NF-AI: Identifying Neurofibromatosis Tumors using Liquid Neural Network (LTC) trained on AACR GENIE Datasets [0.0]
  We propose an interpretable AI approach to diagnose patients with neurofibromatosis. Our proposed approach outperformed existing models with 99.86% accuracy.
  arXiv  Detail & Related papers  (2023-04-26T10:28:59Z)
• A Brief Review of Explainable Artificial Intelligence in Healthcare [7.844015105790313]
  XAI refers to the techniques and methods for building AI applications. Model explainability and interpretability are vital successful deployment of AI models in healthcare practices.
  arXiv  Detail & Related papers  (2023-04-04T05:41:57Z)
• Safe AI for health and beyond -- Monitoring to transform a health service [51.8524501805308]
  We will assess the infrastructure required to monitor the outputs of a machine learning algorithm. We will present two scenarios with examples of monitoring and updates of models.
  arXiv  Detail & Related papers  (2023-03-02T17:27:45Z)
• Context-dependent Explainability and Contestability for Trustworthy Medical Artificial Intelligence: Misclassification Identification of Morbidity Recognition Models in Preterm Infants [0.0]
  Explainable AI (XAI) aims to address this requirement by clarifying AI reasoning to support the end users. We built our methodology on three main pillars: decomposing the feature set by leveraging clinical context latent space, assessing the clinical association of global explanations, and Latent Space Similarity (LSS) based local explanations.
  arXiv  Detail & Related papers  (2022-12-17T07:59:09Z)
• Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing [62.9062883851246]
  Machine learning holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. Using multi-task learning, we propose the first method to assess and mitigate shortcut learning as a part of the fairness assessment of clinical ML systems.
  arXiv  Detail & Related papers  (2022-07-21T09:35:38Z)
• Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability [82.29775890542967]
  Estimating personalized effects of treatments is a complex, yet pervasive problem. Recent developments in the machine learning literature on heterogeneous treatment effect estimation gave rise to many sophisticated, but opaque, tools. We use post-hoc feature importance methods to identify features that influence the model's predictions.
  arXiv  Detail & Related papers  (2022-06-16T17:59:05Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction [55.94378672172967]
  We focus on few-shot disease subtype prediction problem, identifying subgroups of similar patients. We introduce meta learning techniques to develop a new model, which can extract the common experience or knowledge from interrelated clinical tasks. Our new model is built upon a carefully designed meta-learner, called Prototypical Network, that is a simple yet effective meta learning machine for few-shot image classification.
  arXiv  Detail & Related papers  (2020-09-02T02:50:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.