Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluation

• URL: http://arxiv.org/abs/2308.06100v1
• Date: Fri, 11 Aug 2023 12:22:37 GMT
• Title: Diffusion-based Visual Counterfactual Explanations -- Towards Systematic Quantitative Evaluation
• Authors: Philipp Vaeth and Alexander M. Fruehwald and Benjamin Paassen and Magda Gregorova
• Abstract summary: Latest methods for visual counterfactual explanations (VCE) harness the power of deep generative models to synthesize new examples of high-dimensional images of impressive quality. It is currently difficult to compare the performance of these VCE methods as the evaluation procedures largely vary and often boil down to visual inspection of individual examples and small scale user studies. We propose a framework for systematic, quantitative evaluation of the VCE methods and a minimal set of metrics to be used.
• Score: 64.0476282000118
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latest methods for visual counterfactual explanations (VCE) harness the power of deep generative models to synthesize new examples of high-dimensional images of impressive quality. However, it is currently difficult to compare the performance of these VCE methods as the evaluation procedures largely vary and often boil down to visual inspection of individual examples and small scale user studies. In this work, we propose a framework for systematic, quantitative evaluation of the VCE methods and a minimal set of metrics to be used. We use this framework to explore the effects of certain crucial design choices in the latest diffusion-based generative models for VCEs of natural image classification (ImageNet). We conduct a battery of ablation-like experiments, generating thousands of VCEs for a suite of classifiers of various complexity, accuracy and robustness. Our findings suggest multiple directions for future advancements and improvements of VCE methods. By sharing our methodology and our approach to tackle the computational challenges of such a study on a limited hardware setup (including the complete code base), we offer a valuable guidance for researchers in the field fostering consistency and transparency in the assessment of counterfactual explanations.

Related papers

• Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation [37.22528391940295]
  We propose JD-CCL (Jaccard Distance-based Contrastive Learning), a novel approach to enhance the ability to match multimodal entity linking models. To address the limitations caused by the variations within the visual modality among mentions and entities, we introduce a novel method, CVaCPT (Con Visual-aid Controllable Patch Transform)
  arXiv  Detail & Related papers  (2025-01-24T01:35:10Z)
• DepthMamba with Adaptive Fusion [0.0]
  We propose a new robustness benchmark to evaluate the depth estimation system under various noisy pose settings. To tackle this challenge, we propose a two-branch network architecture which fuses the depth estimation results of single-view and multi-view branch. The proposed method can perform well on some challenging scenes including dynamic objects, texture-less regions, etc.
  arXiv  Detail & Related papers  (2024-12-28T01:17:47Z)
• A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends [67.43992456058541]
  Image restoration (IR) refers to the process of improving visual quality of images while removing degradation, such as noise, blur, weather effects, and so on. Traditional IR methods typically target specific types of degradation, which limits their effectiveness in real-world scenarios with complex distortions. The all-in-one image restoration (AiOIR) paradigm has emerged, offering a unified framework that adeptly addresses multiple degradation types.
  arXiv  Detail & Related papers  (2024-10-19T11:11:09Z)
• Preview-based Category Contrastive Learning for Knowledge Distillation [53.551002781828146]
  We propose a novel preview-based category contrastive learning method for knowledge distillation (PCKD) It first distills the structural knowledge of both instance-level feature correspondence and the relation between instance features and category centers. It can explicitly optimize the category representation and explore the distinct correlation between representations of instances and categories.
  arXiv  Detail & Related papers  (2024-10-18T03:31:00Z)
• Deep Learning for Video Anomaly Detection: A Review [52.74513211976795]
  Video anomaly detection (VAD) aims to discover behaviors or events deviating from the normality in videos. In the era of deep learning, a great variety of deep learning based methods are constantly emerging for the VAD task. This review covers the spectrum of five different categories, namely, semi-supervised, weakly supervised, fully supervised, unsupervised and open-set supervised VAD.
  arXiv  Detail & Related papers  (2024-09-09T07:31:16Z)
• Better Understanding Differences in Attribution Methods via Systematic Evaluations [57.35035463793008]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models.
  arXiv  Detail & Related papers  (2023-03-21T14:24:58Z)
• On the Effects of Self-supervision and Contrastive Alignment in Deep Multi-view Clustering [16.63376980974536]
  We present a unified framework for deep MVC that includes many recent methods as instances. We make key observations about the effect of self-supervision, and in particular, drawbacks of aligning representations with contrastive learning. Motivated by our findings, we develop several new DeepMVC instances with new forms of self-supervision.
  arXiv  Detail & Related papers  (2023-03-17T10:51:38Z)
• Towards Better Understanding Attribution Methods [77.1487219861185]
  Post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. We propose three novel evaluation schemes to more reliably measure the faithfulness of those methods. We also propose a post-processing smoothing step that significantly improves the performance of some attribution methods.
  arXiv  Detail & Related papers  (2022-05-20T20:50:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.