Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models

• URL: http://arxiv.org/abs/2303.12641v2
• Date: Mon, 27 Mar 2023 07:43:17 GMT
• Title: Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models
• Authors: Frederik Pahde, Maximilian Dreyer, Wojciech Samek, Sebastian Lapuschkin
• Abstract summary: State-of-the-art machine learning models often learn spurious correlations embedded in the training data. This poses risks when deploying these models for high-stake decision-making. We propose Reveal to Revise (R2R) to identify, mitigate, and (re-)evaluate spurious model behavior.
• Score: 11.879170124003252
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art machine learning models often learn spurious correlations embedded in the training data. This poses risks when deploying these models for high-stake decision-making, such as in medical applications like skin cancer detection. To tackle this problem, we propose Reveal to Revise (R2R), a framework entailing the entire eXplainable Artificial Intelligence (XAI) life cycle, enabling practitioners to iteratively identify, mitigate, and (re-)evaluate spurious model behavior with a minimal amount of human interaction. In the first step (1), R2R reveals model weaknesses by finding outliers in attributions or through inspection of latent concepts learned by the model. Secondly (2), the responsible artifacts are detected and spatially localized in the input data, which is then leveraged to (3) revise the model behavior. Concretely, we apply the methods of RRR, CDEP and ClArC for model correction, and (4) (re-)evaluate the model's performance and remaining sensitivity towards the artifact. Using two medical benchmark datasets for Melanoma detection and bone age estimation, we apply our R2R framework to VGG, ResNet and EfficientNet architectures and thereby reveal and correct real dataset-intrinsic artifacts, as well as synthetic variants in a controlled setting. Completing the XAI life cycle, we demonstrate multiple R2R iterations to mitigate different biases. Code is available on https://github.com/maxdreyer/Reveal2Revise.

Related papers

• Explainable AI for Comparative Analysis of Intrusion Detection Models [20.683181384051395]
  This research analyzes various machine learning models to the tasks of binary and multi-class classification for intrusion detection from network traffic. We trained all models to the accuracy of 90% on the UNSW-NB15 dataset. We also discover that Random Forest provides the best performance in terms of accuracy, time efficiency and robustness.
  arXiv  Detail & Related papers  (2024-06-14T03:11:01Z)
• Investigating the Robustness of Counterfactual Learning to Rank Models: A Reproducibility Study [61.64685376882383]
  Counterfactual learning to rank (CLTR) has attracted extensive attention in the IR community for its ability to leverage massive logged user interaction data to train ranking models. This paper investigates the robustness of existing CLTR models in complex and diverse situations. We find that the DLA models and IPS-DCM show better robustness under various simulation settings than IPS-PBM and PRS with offline propensity estimation.
  arXiv  Detail & Related papers  (2024-04-04T10:54:38Z)
• A Study on Domain Generalization for Failure Detection through Human Reactions in HRI [7.664159325276515]
  Machine learning models are commonly tested in-distribution (same dataset); performance almost always drops in out-of-distribution settings. This makes domain generalization - retaining performance in different settings - a critical issue. We present a concise analysis of domain generalization in failure detection models trained on human facial expressions.
  arXiv  Detail & Related papers  (2024-03-10T21:30:22Z)
• Spot The Odd One Out: Regularized Complete Cycle Consistent Anomaly Detector GAN [4.5123329001179275]
  This study presents an adversarial method for anomaly detection in real-world applications, leveraging the power of generative adversarial neural networks (GANs) Previous methods suffer from the high variance between class-wise accuracy which leads to not being applicable for all types of anomalies. The proposed method named RCALAD tries to solve this problem by introducing a novel discriminator to the structure, which results in a more efficient training process.
  arXiv  Detail & Related papers  (2023-04-16T13:05:39Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• CARLA-GeAR: a Dataset Generator for a Systematic Evaluation of Adversarial Robustness of Vision Models [61.68061613161187]
  This paper presents CARLA-GeAR, a tool for the automatic generation of synthetic datasets for evaluating the robustness of neural models against physical adversarial patches. The tool is built on the CARLA simulator, using its Python API, and allows the generation of datasets for several vision tasks in the context of autonomous driving. The paper presents an experimental study to evaluate the performance of some defense methods against such attacks, showing how the datasets generated with CARLA-GeAR might be used in future work as a benchmark for adversarial defense in the real world.
  arXiv  Detail & Related papers  (2022-06-09T09:17:38Z)
• X-model: Improving Data Efficiency in Deep Learning with A Minimax Model [78.55482897452417]
  We aim at improving data efficiency for both classification and regression setups in deep learning. To take the power of both worlds, we propose a novel X-model. X-model plays a minimax game between the feature extractor and task-specific heads.
  arXiv  Detail & Related papers  (2021-10-09T13:56:48Z)
• Disentangled Recurrent Wasserstein Autoencoder [17.769077848342334]
  recurrent Wasserstein Autoencoder (R-WAE) is a new framework for generative modeling of sequential data. R-WAE disentangles the representation of an input sequence into static and dynamic factors. Our models outperform other baselines with the same settings in terms of disentanglement and unconditional video generation.
  arXiv  Detail & Related papers  (2021-01-19T07:43:25Z)
• Probing Model Signal-Awareness via Prediction-Preserving Input Minimization [67.62847721118142]
  We evaluate models' ability to capture the correct vulnerability signals to produce their predictions. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR) The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric.
  arXiv  Detail & Related papers  (2020-11-25T20:05:23Z)
• A Spatial-Temporal Graph Based Hybrid Infectious Disease Model with Application to COVID-19 [3.785123406103385]
  As the COVID-19 pandemic evolves, reliable prediction plays an important role for policy making. The data-driven machine learning models such as RNN can suffer in case of limited time series data such as COVID-19. We combine SEIR and RNN on a graph structure to develop a hybrid-temporal model to achieve both accuracy and efficiency in training and forecasting.
  arXiv  Detail & Related papers  (2020-10-18T19:34:54Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.