Diffusion-based Visual Counterfactual Explanations -- Towards Systematic
Quantitative Evaluation
- URL: http://arxiv.org/abs/2308.06100v1
- Date: Fri, 11 Aug 2023 12:22:37 GMT
- Title: Diffusion-based Visual Counterfactual Explanations -- Towards Systematic
Quantitative Evaluation
- Authors: Philipp Vaeth and Alexander M. Fruehwald and Benjamin Paassen and
Magda Gregorova
- Abstract summary: Latest methods for visual counterfactual explanations (VCE) harness the power of deep generative models to synthesize new examples of high-dimensional images of impressive quality.
It is currently difficult to compare the performance of these VCE methods as the evaluation procedures largely vary and often boil down to visual inspection of individual examples and small scale user studies.
We propose a framework for systematic, quantitative evaluation of the VCE methods and a minimal set of metrics to be used.
- Score: 64.0476282000118
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Latest methods for visual counterfactual explanations (VCE) harness the power
of deep generative models to synthesize new examples of high-dimensional images
of impressive quality. However, it is currently difficult to compare the
performance of these VCE methods as the evaluation procedures largely vary and
often boil down to visual inspection of individual examples and small scale
user studies. In this work, we propose a framework for systematic, quantitative
evaluation of the VCE methods and a minimal set of metrics to be used. We use
this framework to explore the effects of certain crucial design choices in the
latest diffusion-based generative models for VCEs of natural image
classification (ImageNet). We conduct a battery of ablation-like experiments,
generating thousands of VCEs for a suite of classifiers of various complexity,
accuracy and robustness. Our findings suggest multiple directions for future
advancements and improvements of VCE methods. By sharing our methodology and
our approach to tackle the computational challenges of such a study on a
limited hardware setup (including the complete code base), we offer a valuable
guidance for researchers in the field fostering consistency and transparency in
the assessment of counterfactual explanations.
Related papers
- Opinion-Unaware Blind Image Quality Assessment using Multi-Scale Deep Feature Statistics [54.08757792080732]
We propose integrating deep features from pre-trained visual models with a statistical analysis model to achieve opinion-unaware BIQA (OU-BIQA)
Our proposed model exhibits superior consistency with human visual perception compared to state-of-the-art BIQA models.
arXiv Detail & Related papers (2024-05-29T06:09:34Z) - Multi-Modal Prompt Learning on Blind Image Quality Assessment [65.0676908930946]
Image Quality Assessment (IQA) models benefit significantly from semantic information, which allows them to treat different types of objects distinctly.
Traditional methods, hindered by a lack of sufficiently annotated data, have employed the CLIP image-text pretraining model as their backbone to gain semantic awareness.
Recent approaches have attempted to address this mismatch using prompt technology, but these solutions have shortcomings.
This paper introduces an innovative multi-modal prompt-based methodology for IQA.
arXiv Detail & Related papers (2024-04-23T11:45:32Z) - Semi-supervised multi-view concept decomposition [30.699496411869834]
Concept Factorization (CF) has demonstrated superior performance in multi-view clustering tasks.
We propose a novel semi-supervised multi-view concept factorization model, named SMVCF.
We conduct experiments on four diverse datasets to evaluate the performance of SMVCF.
arXiv Detail & Related papers (2023-07-03T10:50:44Z) - Learning from Multi-Perception Features for Real-Word Image
Super-resolution [87.71135803794519]
We propose a novel SR method called MPF-Net that leverages multiple perceptual features of input images.
Our method incorporates a Multi-Perception Feature Extraction (MPFE) module to extract diverse perceptual information.
We also introduce a contrastive regularization term (CR) that improves the model's learning capability.
arXiv Detail & Related papers (2023-05-26T07:35:49Z) - Rank Flow Embedding for Unsupervised and Semi-Supervised Manifold
Learning [9.171175292808144]
We propose a novel manifold learning algorithm named Rank Flow Embedding (RFE) for unsupervised and semi-supervised scenarios.
RFE computes context-sensitive embeddings, which are refined following a rank-based processing flow.
The generated embeddings can be exploited for more effective unsupervised retrieval or semi-supervised classification.
arXiv Detail & Related papers (2023-04-24T21:02:12Z) - Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions.
We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods.
We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
arXiv Detail & Related papers (2023-03-21T14:24:58Z) - On the Effects of Self-supervision and Contrastive Alignment in Deep
Multi-view Clustering [16.63376980974536]
We present a unified framework for deep MVC that includes many recent methods as instances.
We make key observations about the effect of self-supervision, and in particular, drawbacks of aligning representations with contrastive learning.
Motivated by our findings, we develop several new DeepMVC instances with new forms of self-supervision.
arXiv Detail & Related papers (2023-03-17T10:51:38Z) - Towards Better Understanding Attribution Methods [77.1487219861185]
Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions.
We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods.
We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
arXiv Detail & Related papers (2022-05-20T20:50:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.