Clarity: an improved gradient method for producing quality visual
counterfactual explanations
- URL: http://arxiv.org/abs/2211.15370v1
- Date: Tue, 22 Nov 2022 10:53:17 GMT
- Title: Clarity: an improved gradient method for producing quality visual
counterfactual explanations
- Authors: Claire Theobald, Fr\'ed\'eric Pennerath, Brieuc Conan-Guez, Miguel
Couceiro, Amedeo Napoli
- Abstract summary: Visual counterfactual explanations identify modifications to an image that would change the prediction of a classifier.
We propose techniques based on generative models (VAE) and a classifier ensemble directly trained in the latent space.
These improvements lead to a novel classification model, Clarity, which produces realistic counterfactual explanations over all images.
- Score: 7.279730418361996
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Visual counterfactual explanations identify modifications to an image that
would change the prediction of a classifier. We propose a set of techniques
based on generative models (VAE) and a classifier ensemble directly trained in
the latent space, which all together, improve the quality of the gradient
required to compute visual counterfactuals. These improvements lead to a novel
classification model, Clarity, which produces realistic counterfactual
explanations over all images. We also present several experiments that give
insights on why these techniques lead to better quality results than those in
the literature. The explanations produced are competitive with the
state-of-the-art and emphasize the importance of selecting a meaningful input
space for training.
Related papers
- CoLa-DCE -- Concept-guided Latent Diffusion Counterfactual Explanations [2.3083192626377755]
We introduce Concept-guided Latent Diffusion Counterfactual Explanations (CoLa-DCE)
CoLa-DCE generates concept-guided counterfactuals for any classifier with a high degree of control regarding concept selection and spatial conditioning.
We demonstrate the advantages of our approach in minimality and comprehenibility across multiple image classification models and datasets.
arXiv Detail & Related papers (2024-06-03T14:27:46Z) - Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly.
Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness.
Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings.
This paper introduces an innovative multi-modal prompt-based methodology for IQA.
arXiv Detail & Related papers (2024-04-23T11:45:32Z) - Text Descriptions are Compressive and Invariant Representations for
Visual Learning [63.3464863723631]
We show that an alternative approach, in line with humans' understanding of multiple visual features per class, can provide compelling performance in the robust few-shot learning setting.
In particular, we introduce a novel method, textit SLR-AVD (Sparse Logistic Regression using Augmented Visual Descriptors).
This method first automatically generates multiple visual descriptions of each class via a large language model (LLM), then uses a VLM to translate these descriptions to a set of visual feature embeddings of each image, and finally uses sparse logistic regression to select a relevant subset of these features to classify
arXiv Detail & Related papers (2023-07-10T03:06:45Z) - SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for
Few-shot Image Classification [84.05253637260743]
We propose a new framework, named Semantic-guided Visual Adapting (SgVA), to extend vision-language pre-trained models.
SgVA produces discriminative task-specific visual features by comprehensively using a vision-specific contrastive loss, a cross-modal contrastive loss, and an implicit knowledge distillation.
State-of-the-art results on 13 datasets demonstrate that the adapted visual features can well complement the cross-modal features to improve few-shot image classification.
arXiv Detail & Related papers (2022-11-28T14:58:15Z) - Explaining Image Classifiers Using Contrastive Counterfactuals in
Generative Latent Spaces [12.514483749037998]
We introduce a novel method to generate causal and yet interpretable counterfactual explanations for image classifiers.
We use this framework to obtain contrastive and causal sufficiency and necessity scores as global explanations for black-box classifiers.
arXiv Detail & Related papers (2022-06-10T17:54:46Z) - Improving the trustworthiness of image classification models by
utilizing bounding-box annotations [16.936384403276925]
We propose to optimize a training objective that incorporates bounding box information, which is available in many image classification datasets.
Preliminary experimental results show that the proposed algorithm achieves better performance in accuracy, robustness, and interpretability compared with baselines.
arXiv Detail & Related papers (2021-08-15T15:09:07Z) - Revisiting The Evaluation of Class Activation Mapping for
Explainability: A Novel Metric and Experimental Analysis [54.94682858474711]
Class Activation Mapping (CAM) approaches provide an effective visualization by taking weighted averages of the activation maps.
We propose a novel set of metrics to quantify explanation maps, which show better effectiveness and simplify comparisons between approaches.
arXiv Detail & Related papers (2021-04-20T21:34:24Z) - This is not the Texture you are looking for! Introducing Novel
Counterfactual Explanations for Non-Experts using Generative Adversarial
Learning [59.17685450892182]
counterfactual explanation systems try to enable a counterfactual reasoning by modifying the input image.
We present a novel approach to generate such counterfactual image explanations based on adversarial image-to-image translation techniques.
Our results show that our approach leads to significantly better results regarding mental models, explanation satisfaction, trust, emotions, and self-efficacy than two state-of-the art systems.
arXiv Detail & Related papers (2020-12-22T10:08:05Z) - A Diagnostic Study of Explainability Techniques for Text Classification [52.879658637466605]
We develop a list of diagnostic properties for evaluating existing explainability techniques.
We compare the saliency scores assigned by the explainability techniques with human annotations of salient input regions to find relations between a model's performance and the agreement of its rationales with human ones.
arXiv Detail & Related papers (2020-09-25T12:01:53Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.