Teaching Humans Subtle Differences with DIFFusion

• URL: http://arxiv.org/abs/2504.08046v1
• Date: Thu, 10 Apr 2025 18:04:22 GMT
• Title: Teaching Humans Subtle Differences with DIFFusion
• Authors: Mia Chiquier, Orr Avrech, Yossi Gandelsman, Berthy Feng, Katherine Bouman, Carl Vondrick,
• Abstract summary: We propose a new method to teach novices how to differentiate between nuanced categories in specialized domains.<n>Our method uses generative models to visualize the minimal change in features to transition between classes.<n> Experiments across six domains show accurate transitions even with limited and unpaired examples.
• Score: 36.30462318766868
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Human expertise depends on the ability to recognize subtle visual differences, such as distinguishing diseases, species, or celestial phenomena. We propose a new method to teach novices how to differentiate between nuanced categories in specialized domains. Our method uses generative models to visualize the minimal change in features to transition between classes, i.e., counterfactuals, and performs well even in domains where data is sparse, examples are unpaired, and category boundaries are not easily explained by text. By manipulating the conditioning space of diffusion models, our proposed method DIFFusion disentangles category structure from instance identity, enabling high-fidelity synthesis even in challenging domains. Experiments across six domains show accurate transitions even with limited and unpaired examples across categories. User studies confirm that our generated counterfactuals outperform unpaired examples in teaching perceptual expertise, showing the potential of generative models for specialized visual learning.

Related papers

• DiffEx: Explaining a Classifier with Diffusion Models to Identify Microscopic Cellular Variations [0.815557531820863]
  Discriminative deep learning models have excelled at classifying images into categories.<n>These models are often perceived as black boxes due to their complexity and lack of interpretability.<n>We propose DiffEx, a method for generating visually interpretable attributes to explain classifiers.
  arXiv  Detail & Related papers  (2025-02-12T12:46:58Z)
• Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Prompts [68.48103545146127]
  This paper proposes a novel framework for unsupervised exploration of diffusion latent spaces. We directly leverage natural language prompts and image captions to map latent directions. Our method provides a more scalable and interpretable understanding of the semantic knowledge encoded within diffusion models.
  arXiv  Detail & Related papers  (2024-10-25T21:44:51Z)
• Diff-Def: Diffusion-Generated Deformation Fields for Conditional Atlases [13.440406411539987]
  Conditional atlases allow for the investigation of fine-grained anatomical differences. We use latent diffusion models to generate deformation fields, which transform a general population atlas into a specific sub-population. We compare our method to several state-of-the-art atlas generation methods in experiments using 5000 brain as well as whole-body MR images from UK Biobank.
  arXiv  Detail & Related papers  (2024-03-25T13:52:48Z)
• NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models [6.254873489691852]
  We propose an unsupervised method to discover latent semantics in text-to-image diffusion models without relying on text prompts. Our method achieves highly disentangled edits, outperforming existing approaches in both diffusion-based and GAN-based latent space editing methods.
  arXiv  Detail & Related papers  (2023-12-08T22:04:53Z)
• Finding Interpretable Class-Specific Patterns through Efficient Neural Search [43.454121220860564]
  We propose a novel, inherently interpretable binary neural network architecture DNAPS that extracts differential patterns from data. DiffNaps is scalable to hundreds of thousands of features and robust to noise. We show on synthetic and real world data, including three biological applications, that, unlike its competitors, DiffNaps consistently yields accurate, succinct, and interpretable class descriptions.
  arXiv  Detail & Related papers  (2023-12-07T14:09:18Z)
• Effective Data Augmentation With Diffusion Models [65.09758931804478]
  We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.
  arXiv  Detail & Related papers  (2023-02-07T20:42:28Z)
• Disentangled representations: towards interpretation of sex determination from hip bone [1.0775419935941009]
  saliency maps have become a popular method to make neural networks interpretable. We propose a new paradigm for better interpretability. We illustrate the relevance of this approach in the context of automatic sex determination from hip bones in forensic medicine.
  arXiv  Detail & Related papers  (2021-12-17T10:07:05Z)
• Discriminative Attribution from Counterfactuals [64.94009515033984]
  We present a method for neural network interpretability by combining feature attribution with counterfactual explanations. We show that this method can be used to quantitatively evaluate the performance of feature attribution methods in an objective manner.
  arXiv  Detail & Related papers  (2021-09-28T00:53:34Z)
• Cross-Domain Similarity Learning for Face Recognition in Unseen Domains [90.35908506994365]
  We introduce a novel cross-domain metric learning loss, which we dub Cross-Domain Triplet (CDT) loss, to improve face recognition in unseen domains. The CDT loss encourages learning semantically meaningful features by enforcing compact feature clusters of identities from one domain. Our method does not require careful hard-pair sample mining and filtering strategy during training.
  arXiv  Detail & Related papers  (2021-03-12T19:48:01Z)
• Learning What Makes a Difference from Counterfactual Examples and Gradient Supervision [57.14468881854616]
  We propose an auxiliary training objective that improves the generalization capabilities of neural networks. We use pairs of minimally-different examples with different labels, a.k.a counterfactual or contrasting examples, which provide a signal indicative of the underlying causal structure of the task. Models trained with this technique demonstrate improved performance on out-of-distribution test sets.
  arXiv  Detail & Related papers  (2020-04-20T02:47:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.