Diffusion Counterfactuals for Image Regressors

• URL: http://arxiv.org/abs/2503.20595v1
• Date: Wed, 26 Mar 2025 14:42:46 GMT
• Title: Diffusion Counterfactuals for Image Regressors
• Authors: Trung Duc Ha, Sidney Bender,
• Abstract summary: We present two methods to create counterfactual explanations for image regression tasks using diffusion-based generative models.<n>Both produce realistic, semantic, and smooth counterfactuals on CelebA-HQ and a synthetic data set.<n>We find that for regression counterfactuals, changes in features depend on the region of the predicted value.
• Score: 1.534667887016089
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Counterfactual explanations have been successfully applied to create human interpretable explanations for various black-box models. They are handy for tasks in the image domain, where the quality of the explanations benefits from recent advances in generative models. Although counterfactual explanations have been widely applied to classification models, their application to regression tasks remains underexplored. We present two methods to create counterfactual explanations for image regression tasks using diffusion-based generative models to address challenges in sparsity and quality: 1) one based on a Denoising Diffusion Probabilistic Model that operates directly in pixel-space and 2) another based on a Diffusion Autoencoder operating in latent space. Both produce realistic, semantic, and smooth counterfactuals on CelebA-HQ and a synthetic data set, providing easily interpretable insights into the decision-making process of the regression model and reveal spurious correlations. We find that for regression counterfactuals, changes in features depend on the region of the predicted value. Large semantic changes are needed for significant changes in predicted values, making it harder to find sparse counterfactuals than with classifiers. Moreover, pixel space counterfactuals are more sparse while latent space counterfactuals are of higher quality and allow bigger semantic changes.

Related papers

• Unifying Image Counterfactuals and Feature Attributions with Latent-Space Adversarial Attacks [3.8642937395065124]
  We introduce a new, easy-to-implement framework for counterfactual images. Our method resembles an adversarial attack on the representation of the image along a low-dimensional manifold. We show how to accompany counterfactuals with feature attribution that quantify the changes between the original and counterfactual images.
  arXiv  Detail & Related papers  (2025-04-21T23:09:30Z)
• MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion [14.907473847787541]
  We propose Masked Diffusion Conditional (MacDiff) as a unified framework for human skeleton modeling. For the first time, we leverage diffusion models as effective skeleton representation learners. MacDiff achieves state-of-the-art performance on representation learning benchmarks while maintaining the competence for generative tasks.
  arXiv  Detail & Related papers  (2024-09-16T17:06:10Z)
• Latent diffusion models for parameterization and data assimilation of facies-based geomodels [0.0]
  Diffusion models are trained to generate new geological realizations from input fields characterized by random noise. Latent diffusion models are shown to provide realizations that are visually consistent with samples from geomodeling software.
  arXiv  Detail & Related papers  (2024-06-21T01:32:03Z)
• Effective Data Augmentation With Diffusion Models [65.09758931804478]
  We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.
  arXiv  Detail & Related papers  (2023-02-07T20:42:28Z)
• Diffusion Visual Counterfactual Explanations [51.077318228247925]
  Visual Counterfactual Explanations (VCEs) are an important tool to understand the decisions of an image. Current approaches for the generation of VCEs are restricted to adversarially robust models and often contain non-realistic artefacts. In this paper, we overcome this by generating Visual Diffusion Counterfactual Explanations (DVCEs) for arbitrary ImageNet classifiers.
  arXiv  Detail & Related papers  (2022-10-21T09:35:47Z)
• X-model: Improving Data Efficiency in Deep Learning with A Minimax Model [78.55482897452417]
  We aim at improving data efficiency for both classification and regression setups in deep learning. To take the power of both worlds, we propose a novel X-model. X-model plays a minimax game between the feature extractor and task-specific heads.
  arXiv  Detail & Related papers  (2021-10-09T13:56:48Z)
• Designing Counterfactual Generators using Deep Model Inversion [31.1607056675927]
  We develop a deep inversion approach to generate counterfactual explanations for a given query image. We find that, in addition to producing visually meaningful explanations, the counterfactuals from DISC are effective at learning decision boundaries and are robust to unknown test-time corruptions.
  arXiv  Detail & Related papers  (2021-09-29T08:40:50Z)
• Discrete Auto-regressive Variational Attention Models for Text Modeling [53.38382932162732]
  Variational autoencoders (VAEs) have been widely applied for text modeling. They are troubled by two challenges: information underrepresentation and posterior collapse. We propose Discrete Auto-regressive Variational Attention Model (DAVAM) to address the challenges.
  arXiv  Detail & Related papers  (2021-06-16T06:36:26Z)
• Beyond Trivial Counterfactual Explanations with Diverse Valuable Explanations [64.85696493596821]
  In computer vision applications, generative counterfactual methods indicate how to perturb a model's input to change its prediction. We propose a counterfactual method that learns a perturbation in a disentangled latent space that is constrained using a diversity-enforcing loss. Our model improves the success rate of producing high-quality valuable explanations when compared to previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-03-18T12:57:34Z)
• Deducing neighborhoods of classes from a fitted model [68.8204255655161]
  In this article a new kind of interpretable machine learning method is presented. It can help to understand the partitioning of the feature space into predicted classes in a classification model using quantile shifts. Basically, real data points (or specific points of interest) are used and the changes of the prediction after slightly raising or decreasing specific features are observed.
  arXiv  Detail & Related papers  (2020-09-11T16:35:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.