LesionGen: A Concept-Guided Diffusion Model for Dermatology Image Synthesis

• URL: http://arxiv.org/abs/2507.23001v1
• Date: Wed, 30 Jul 2025 18:07:34 GMT
• Title: LesionGen: A Concept-Guided Diffusion Model for Dermatology Image Synthesis
• Authors: Jamil Fayyad, Nourhan Bayasi, Ziyang Yu, Homayoun Najjaran,
• Abstract summary: We introduce LesionGen, a clinically informed T2I-DPM framework for dermatology image synthesis.<n>LesionGen is trained on structured, concept-rich dermatological captions derived from expert annotations and pseudo-generated, concept-guided reports.<n>Our results demonstrate that models trained solely on our synthetic dataset achieve classification accuracy comparable to those trained on real images.
• Score: 4.789822624169502
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Deep learning models for skin disease classification require large, diverse, and well-annotated datasets. However, such resources are often limited due to privacy concerns, high annotation costs, and insufficient demographic representation. While text-to-image diffusion probabilistic models (T2I-DPMs) offer promise for medical data synthesis, their use in dermatology remains underexplored, largely due to the scarcity of rich textual descriptions in existing skin image datasets. In this work, we introduce LesionGen, a clinically informed T2I-DPM framework for dermatology image synthesis. Unlike prior methods that rely on simplistic disease labels, LesionGen is trained on structured, concept-rich dermatological captions derived from expert annotations and pseudo-generated, concept-guided reports. By fine-tuning a pretrained diffusion model on these high-quality image-caption pairs, we enable the generation of realistic and diverse skin lesion images conditioned on meaningful dermatological descriptions. Our results demonstrate that models trained solely on our synthetic dataset achieve classification accuracy comparable to those trained on real images, with notable gains in worst-case subgroup performance. Code and data are available here.

Related papers

• Doctor Approved: Generating Medically Accurate Skin Disease Images through AI-Expert Feedback [43.1078084014722]
  We propose a novel framework, coined MAGIC, that synthesizes clinically accurate skin disease images for data augmentation.<n>Our method creatively translates expert-defined criteria into actionable feedback for image synthesis of DMs.
  arXiv  Detail & Related papers  (2025-06-14T03:15:09Z)
• Causal Disentanglement for Robust Long-tail Medical Image Generation [80.15257897500578]
  We propose a novel medical image generation framework, which generates independent pathological and structural features.<n>We leverage a diffusion model guided by pathological findings to model pathological features, enabling the generation of diverse counterfactual images.
  arXiv  Detail & Related papers  (2025-04-20T01:54:18Z)
• Latent Drifting in Diffusion Models for Counterfactual Medical Image Synthesis [55.959002385347645]
  Latent Drifting enables diffusion models to be conditioned for medical images fitted for the complex task of counterfactual image generation.<n>We evaluate our method on three public longitudinal benchmark datasets of brain MRI and chest X-rays for counterfactual image generation.
  arXiv  Detail & Related papers  (2024-12-30T01:59:34Z)
• FairSkin: Fair Diffusion for Skin Disease Image Generation [54.29840149709033]
  Diffusion Model (DM) has become a leading method in generating synthetic medical images, but it suffers from a critical twofold bias. We propose FairSkin, a novel DM framework that mitigates these biases through a three-level resampling mechanism. Our approach significantly improves the diversity and quality of generated images, contributing to more equitable skin disease detection in clinical settings.
  arXiv  Detail & Related papers  (2024-10-29T21:37:03Z)
• MediSyn: A Generalist Text-Guided Latent Diffusion Model For Diverse Medical Image Synthesis [4.541407789437896]
  MediSyn is a text-guided latent diffusion model capable of generating synthetic images from 6 medical specialties and 10 image types.<n>A direct comparison of the synthetic images against the real images confirms that our model synthesizes novel images and, crucially, may preserve patient privacy.<n>Our findings highlight the immense potential for generalist image generative models to accelerate algorithmic research and development in medicine.
  arXiv  Detail & Related papers  (2024-05-16T04:28:44Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Models [49.95603725998561]
  We propose a new paradigm to build robust and interpretable medical image classifiers with natural language concepts. Specifically, we first query clinical concepts from GPT-4, then transform latent image features into explicit concepts with a vision-language model.
  arXiv  Detail & Related papers  (2023-10-04T21:57:09Z)
• Improving dermatology classifiers across populations using images generated by large diffusion models [4.291548465691441]
  We show that DALL$cdot$E 2, a large-scale text-to-image diffusion model, can produce photorealistic images of skin disease across skin types. We demonstrate that augmenting training data with DALL$cdot$E 2-generated synthetic images improves classification of skin disease overall and especially for underrepresented groups.
  arXiv  Detail & Related papers  (2022-11-23T23:53:03Z)
• Analysis of skin lesion images with deep learning [0.0]
  We evaluate the current state of the art in the classification of dermoscopic images. Various deep neural network architectures pre-trained on the ImageNet data set are adapted to a combined training data set. The performance and applicability of these models for the detection of eight classes of skin lesions are examined.
  arXiv  Detail & Related papers  (2021-01-11T10:58:36Z)
• Semi-supervised Medical Image Classification with Relation-driven Self-ensembling Model [71.80319052891817]
  We present a relation-driven semi-supervised framework for medical image classification. It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations. Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
  arXiv  Detail & Related papers  (2020-05-15T06:57:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.