Ensuring Medical AI Safety: Explainable AI-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data

• URL: http://arxiv.org/abs/2501.13818v1
• Date: Thu, 23 Jan 2025 16:39:09 GMT
• Title: Ensuring Medical AI Safety: Explainable AI-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data
• Authors: Frederik Pahde, Thomas Wiegand, Sebastian Lapuschkin, Wojciech Samek,
• Abstract summary: We introduce a semi-automated framework for the identification of spurious behavior from both data and model perspective.<n>This allows the retrieval of spurious data points and the detection of model circuits that encode the associated prediction rules.<n>We show the applicability of our framework using four medical datasets, featuring controlled and real-world spurious correlations.
• Score: 14.991686165405959
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep neural networks are increasingly employed in high-stakes medical applications, despite their tendency for shortcut learning in the presence of spurious correlations, which can have potentially fatal consequences in practice. Detecting and mitigating shortcut behavior is a challenging task that often requires significant labeling efforts from domain experts. To alleviate this problem, we introduce a semi-automated framework for the identification of spurious behavior from both data and model perspective by leveraging insights from eXplainable Artificial Intelligence (XAI). This allows the retrieval of spurious data points and the detection of model circuits that encode the associated prediction rules. Moreover, we demonstrate how these shortcut encodings can be used for XAI-based sample- and pixel-level data annotation, providing valuable information for bias mitigation methods to unlearn the undesired shortcut behavior. We show the applicability of our framework using four medical datasets across two modalities, featuring controlled and real-world spurious correlations caused by data artifacts. We successfully identify and mitigate these biases in VGG16, ResNet50, and contemporary Vision Transformer models, ultimately increasing their robustness and applicability for real-world medical tasks.

Related papers

• Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers [0.0]
  Large language models (LLMs) aligned for safety often exhibit emergent deceptive behaviors.<n>This paper introduces adversarial activation patching, a novel mechanistic interpretability framework.<n>By sourcing activations from "deceptive" prompts, we simulate vulnerabilities and quantify deception rates.
  arXiv  Detail & Related papers  (2025-07-12T21:29:49Z)
• Collaborative Value Function Estimation Under Model Mismatch: A Federated Temporal Difference Analysis [55.13545823385091]
  Federated reinforcement learning (FedRL) enables collaborative learning while preserving data privacy by preventing direct data exchange between agents.<n>In real-world applications, each agent may experience slightly different transition dynamics, leading to inherent model mismatches.<n>We show that even moderate levels of information sharing significantly mitigate environment-specific errors.
  arXiv  Detail & Related papers  (2025-03-21T18:06:28Z)
• Detecting Dataset Bias in Medical AI: A Generalized and Modality-Agnostic Auditing Framework [8.520644988801243]
  latent bias in machine learning datasets can be amplified during training and/or hidden during testing. We present a data modality-agnostic auditing framework for generating targeted hypotheses about sources of bias. We demonstrate the broad applicability and value of our method by analyzing large-scale medical datasets.
  arXiv  Detail & Related papers  (2025-03-13T02:16:48Z)
• Analyzing Adversarial Inputs in Deep Reinforcement Learning [53.3760591018817]
  We present a comprehensive analysis of the characterization of adversarial inputs, through the lens of formal verification. We introduce a novel metric, the Adversarial Rate, to classify models based on their susceptibility to such perturbations. Our analysis empirically demonstrates how adversarial inputs can affect the safety of a given DRL system with respect to such perturbations.
  arXiv  Detail & Related papers  (2024-02-07T21:58:40Z)
• Leveraging Unlabeled Data for 3D Medical Image Segmentation through Self-Supervised Contrastive Learning [3.7395287262521717]
  Current 3D semi-supervised segmentation methods face significant challenges such as limited consideration of contextual information. We introduce two distinctworks designed to explore and exploit the discrepancies between them, ultimately correcting the erroneous prediction results. We employ a self-supervised contrastive learning paradigm to distinguish between reliable and unreliable predictions.
  arXiv  Detail & Related papers  (2023-11-21T14:03:16Z)
• Detecting Spurious Correlations via Robust Visual Concepts in Real and AI-Generated Image Classification [12.992095539058022]
  We introduce a general-purpose method that efficiently detects potential spurious correlations. The proposed method provides intuitive explanations while eliminating the need for pixel-level annotations. Our method is also suitable for detecting spurious correlations that may propagate to downstream applications originating from generative models.
  arXiv  Detail & Related papers  (2023-11-03T01:12:35Z)
• A Discrepancy Aware Framework for Robust Anomaly Detection [51.710249807397695]
  We present a Discrepancy Aware Framework (DAF), which demonstrates robust performance consistently with simple and cheap strategies. Our method leverages an appearance-agnostic cue to guide the decoder in identifying defects, thereby alleviating its reliance on synthetic appearance. Under the simple synthesis strategies, it outperforms existing methods by a large margin. Furthermore, it also achieves the state-of-the-art localization performance.
  arXiv  Detail & Related papers  (2023-10-11T15:21:40Z)
• Improving Explainability of Disentangled Representations using Multipath-Attribution Mappings [12.145748796751619]
  We propose a framework that utilizes interpretable disentangled representations for downstream-task prediction. We demonstrate the effectiveness of our approach on a synthetic benchmark suite and two medical datasets.
  arXiv  Detail & Related papers  (2023-06-15T10:52:29Z)
• Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck [77.37409441129995]
  In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition. We consider an adversarial threat model under a mutual information constraint to cover a wider class of perturbations in training. We propose an autoencoder-based training to implement the objective, as well as practical encoder designs to facilitate the proposed hybrid discriminative-generative training.
  arXiv  Detail & Related papers  (2023-03-24T16:03:21Z)
• Shortcut Detection with Variational Autoencoders [1.3174512123890016]
  We present a novel approach to detect shortcuts in image and audio datasets by leveraging variational autoencoders (VAEs) The disentanglement of features in the latent space of VAEs allows us to discover feature-target correlations in datasets and semi-automatically evaluate them for ML shortcuts. We demonstrate the applicability of our method on several real-world datasets and identify shortcuts that have not been discovered before.
  arXiv  Detail & Related papers  (2023-02-08T18:26:10Z)
• Detecting Adversarial Examples in Batches -- a geometrical approach [0.0]
  We introduce two geometric metrics, density and coverage, and evaluate their use in detecting adversarial samples in batches of unseen data. We empirically study these metrics using MNIST and two real-world biomedical datasets from MedMNIST, subjected to two different adversarial attacks. Our experiments show promising results for both metrics to detect adversarial examples.
  arXiv  Detail & Related papers  (2022-06-17T12:52:43Z)
• Causal Discovery and Knowledge Injection for Contestable Neural Networks (with Appendices) [10.616061367794385]
  We propose a two-way interaction whereby neural-network-empowered machines can expose the underpinning learnt causal graphs. We show that our method improves predictive performance up to 2.4x while producing parsimonious networks, up to 7x smaller in the input layer.
  arXiv  Detail & Related papers  (2022-05-19T18:21:12Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)
• Towards Unbiased Visual Emotion Recognition via Causal Intervention [63.74095927462]
  We propose a novel Emotion Recognition Network (IERN) to alleviate the negative effects brought by the dataset bias. A series of designed tests validate the effectiveness of IERN, and experiments on three emotion benchmarks demonstrate that IERN outperforms other state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-07-26T10:40:59Z)
• Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching [58.26619456972598]
  Federated learning (FL) has emerged with increasing popularity to collaborate distributed medical institutions for training deep networks. This paper studies a practical yet challenging FL problem, named textitFederated Semi-supervised Learning (FSSL) We present a novel approach for this problem, which improves over traditional consistency regularization mechanism with a new inter-client relation matching scheme.
  arXiv  Detail & Related papers  (2021-06-16T07:58:00Z)
• Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [74.05906222376608]
  We propose adversarial self-supervision UDA (or ASSUDA) that maximizes the agreement between clean images and their adversarial examples by a contrastive loss in the output space. This paper is rooted in two observations: (i) the robustness of UDA methods in semantic segmentation remains unexplored, which pose a security concern in this field; and (ii) although commonly used self-supervision (e.g., rotation and jigsaw) benefits image tasks such as classification and recognition, they fail to provide the critical supervision signals that could learn discriminative representation for segmentation tasks.
  arXiv  Detail & Related papers  (2021-05-23T01:50:44Z)
• Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profiles [69.9674326582747]
  This paper presents a visual framework to investigate neural network models subjected to adversarial examples. We show how observing these elements can quickly pinpoint exploited areas in a model.
  arXiv  Detail & Related papers  (2021-03-18T13:04:21Z)
• Estimating and Improving Fairness with Adversarial Learning [65.99330614802388]
  We propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We evaluate our framework on a large-scale public-available skin lesion dataset.
  arXiv  Detail & Related papers  (2021-03-07T03:10:32Z)
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models [103.64435911083432]
  We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI) PPI leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
  arXiv  Detail & Related papers  (2020-12-06T20:30:26Z)
• Unsupervised Multi-Modal Representation Learning for Affective Computing with Multi-Corpus Wearable Data [16.457778420360537]
  We propose an unsupervised framework to reduce the reliance on human supervision. The proposed framework utilizes two stacked convolutional autoencoders to learn latent representations from wearable electrocardiogram (ECG) and electrodermal activity (EDA) signals. Our method outperforms current state-of-the-art results that have performed arousal detection on the same datasets.
  arXiv  Detail & Related papers  (2020-08-24T22:01:55Z)
• Drug-Drug Interaction Prediction with Wasserstein Adversarial Autoencoder-based Knowledge Graph Embeddings [22.562175708415392]
  We propose a new knowledge graph embedding framework for drug-drug interactions. In our framework, the autoencoder is employed to generate high-quality negative samples. The discriminator learns the embeddings of drugs and interactions based on both positive and negative triplets.
  arXiv  Detail & Related papers  (2020-04-15T21:03:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.