Diffusion-based Visual Counterfactual Explanations -- Towards Systematic
Quantitative Evaluation
- URL: http://arxiv.org/abs/2308.06100v1
- Date: Fri, 11 Aug 2023 12:22:37 GMT
- Title: Diffusion-based Visual Counterfactual Explanations -- Towards Systematic
Quantitative Evaluation
- Authors: Philipp Vaeth and Alexander M. Fruehwald and Benjamin Paassen and
Magda Gregorova
- Abstract summary: Latest methods for visual counterfactual explanations (VCE) harness the power of deep generative models to synthesize new examples of high-dimensional images of impressive quality.
It is currently difficult to compare the performance of these VCE methods as the evaluation procedures largely vary and often boil down to visual inspection of individual examples and small scale user studies.
We propose a framework for systematic, quantitative evaluation of the VCE methods and a minimal set of metrics to be used.
- Score: 64.0476282000118
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Latest methods for visual counterfactual explanations (VCE) harness the power
of deep generative models to synthesize new examples of high-dimensional images
of impressive quality. However, it is currently difficult to compare the
performance of these VCE methods as the evaluation procedures largely vary and
often boil down to visual inspection of individual examples and small scale
user studies. In this work, we propose a framework for systematic, quantitative
evaluation of the VCE methods and a minimal set of metrics to be used. We use
this framework to explore the effects of certain crucial design choices in the
latest diffusion-based generative models for VCEs of natural image
classification (ImageNet). We conduct a battery of ablation-like experiments,
generating thousands of VCEs for a suite of classifiers of various complexity,
accuracy and robustness. Our findings suggest multiple directions for future
advancements and improvements of VCE methods. By sharing our methodology and
our approach to tackle the computational challenges of such a study on a
limited hardware setup (including the complete code base), we offer a valuable
guidance for researchers in the field fostering consistency and transparency in
the assessment of counterfactual explanations.
Related papers
- Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation [37.22528391940295]
We propose JD-CCL (Jaccard Distance-based Contrastive Learning), a novel approach to enhance the ability to match multimodal entity linking models.
To address the limitations caused by the variations within the visual modality among mentions and entities, we introduce a novel method, CVaCPT (Con Visual-aid Controllable Patch Transform)
arXiv Detail & Related papers (2025-01-24T01:35:10Z) - DepthMamba with Adaptive Fusion [0.0]
We propose a new robustness benchmark to evaluate the depth estimation system under various noisy pose settings.
To tackle this challenge, we propose a two-branch network architecture which fuses the depth estimation results of single-view and multi-view branch.
The proposed method can perform well on some challenging scenes including dynamic objects, texture-less regions, etc.
arXiv Detail & Related papers (2024-12-28T01:17:47Z) - A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
Image restoration (IR) refers to the process of improving visual quality of images while removing degradation, such as noise, blur, weather effects, and so on.
Traditional IR methods typically target specific types of degradation, which limits their effectiveness in real-world scenarios with complex distortions.
The all-in-one image restoration (AiOIR) paradigm has emerged, offering a unified framework that adeptly addresses multiple degradation types.
arXiv Detail & Related papers (2024-10-19T11:11:09Z) - Preview-based Category Contrastive Learning for Knowledge Distillation [53.551002781828146]
We propose a novel preview-based category contrastive learning method for knowledge distillation (PCKD)
It first distills the structural knowledge of both instance-level feature correspondence and the relation between instance features and category centers.
It can explicitly optimize the category representation and explore the distinct correlation between representations of instances and categories.
arXiv Detail & Related papers (2024-10-18T03:31:00Z) - Deep Learning for Video Anomaly Detection: A Review [52.74513211976795]
Video anomaly detection (VAD) aims to discover behaviors or events deviating from the normality in videos.
In the era of deep learning, a great variety of deep learning based methods are constantly emerging for the VAD task.
This review covers the spectrum of five different categories, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD.
arXiv Detail & Related papers (2024-09-09T07:31:16Z) - Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions.
We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods.
We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
arXiv Detail & Related papers (2023-03-21T14:24:58Z) - On the Effects of Self-supervision and Contrastive Alignment in Deep
Multi-view Clustering [16.63376980974536]
We present a unified framework for deep MVC that includes many recent methods as instances.
We make key observations about the effect of self-supervision, and in particular, drawbacks of aligning representations with contrastive learning.
Motivated by our findings, we develop several new DeepMVC instances with new forms of self-supervision.
arXiv Detail & Related papers (2023-03-17T10:51:38Z) - Towards Better Understanding Attribution Methods [77.1487219861185]
Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions.
We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods.
We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
arXiv Detail & Related papers (2022-05-20T20:50:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.