Controllable Skin Synthesis via Lesion-Focused Vector Autoregression Model

• URL: http://arxiv.org/abs/2508.19626v1
• Date: Wed, 27 Aug 2025 07:04:58 GMT
• Title: Controllable Skin Synthesis via Lesion-Focused Vector Autoregression Model
• Authors: Jiajun Sun, Zhen Yu, Siyuan Yan, Jason J. Ong, Zongyuan Ge, Lei Zhang,
• Abstract summary: The study highlights our controllable skin synthesis model's effectiveness in generating high-fidelity, clinically relevant synthetic skin images.<n>Our method achieves the best overall FID score (average 0.74) among seven lesion types, improving upon the previous state-of-the-art (SOTA) by 6.3%.
• Score: 23.122488283724433
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Skin images from real-world clinical practice are often limited, resulting in a shortage of training data for deep-learning models. While many studies have explored skin image synthesis, existing methods often generate low-quality images and lack control over the lesion's location and type. To address these limitations, we present LF-VAR, a model leveraging quantified lesion measurement scores and lesion type labels to guide the clinically relevant and controllable synthesis of skin images. It enables controlled skin synthesis with specific lesion characteristics based on language prompts. We train a multiscale lesion-focused Vector Quantised Variational Auto-Encoder (VQVAE) to encode images into discrete latent representations for structured tokenization. Then, a Visual AutoRegressive (VAR) Transformer trained on tokenized representations facilitates image synthesis. Lesion measurement from the lesion region and types as conditional embeddings are integrated to enhance synthesis fidelity. Our method achieves the best overall FID score (average 0.74) among seven lesion types, improving upon the previous state-of-the-art (SOTA) by 6.3%. The study highlights our controllable skin synthesis model's effectiveness in generating high-fidelity, clinically relevant synthetic skin images. Our framework code is available at https://github.com/echosun1996/LF-VAR.

Related papers

• A Semantically Enhanced Generative Foundation Model Improves Pathological Image Synthesis [82.01597026329158]
  We introduce a Correlation-Regulated Alignment Framework for Tissue Synthesis (CRAFTS) for pathology-specific text-to-image synthesis.<n>CRAFTS incorporates a novel alignment mechanism that suppresses semantic drift to ensure biological accuracy.<n>This model generates diverse pathological images spanning 30 cancer types, with quality rigorously validated by objective metrics and pathologist evaluations.
  arXiv  Detail & Related papers  (2025-12-15T10:22:43Z)
• SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning [50.69448058071441]
  Deciphering how visual stimuli are transformed into cortical responses is a fundamental challenge in computational neuroscience.<n>We propose SynBrain, a generative framework that simulates the transformation from visual semantics to neural responses.<n>We show that SynBrain surpasses state-of-the-art methods in subject-specific visual-to-fMRI encoding performance.
  arXiv  Detail & Related papers  (2025-08-14T03:01:05Z)
• LesionGen: A Concept-Guided Diffusion Model for Dermatology Image Synthesis [4.789822624169502]
  We introduce LesionGen, a clinically informed T2I-DPM framework for dermatology image synthesis.<n>LesionGen is trained on structured, concept-rich dermatological captions derived from expert annotations and pseudo-generated, concept-guided reports.<n>Our results demonstrate that models trained solely on our synthetic dataset achieve classification accuracy comparable to those trained on real images.
  arXiv  Detail & Related papers  (2025-07-30T18:07:34Z)
• S-SYNTH: Knowledge-Based, Synthetic Generation of Skin Images [2.79604239303318]
  We propose S-SYNTH, the first knowledge-based, adaptable open-source skin simulation framework. We generate synthetic skin, 3D models and digitally rendered images using an anatomically inspired multi-layer, multi-representation skin and growing lesion model. We show that results obtained using synthetic data follow similar comparative trends as real dermatologic images.
  arXiv  Detail & Related papers  (2024-07-31T23:16:29Z)
• Scaling Laws of Synthetic Images for Model Training ... for Now [54.43596959598466]
  We study the scaling laws of synthetic images generated by state of the art text-to-image models. We observe that synthetic images demonstrate a scaling trend similar to, but slightly less effective than, real images in CLIP training.
  arXiv  Detail & Related papers  (2023-12-07T18:59:59Z)
• ContraNeRF: Generalizable Neural Radiance Fields for Synthetic-to-real Novel View Synthesis via Contrastive Learning [102.46382882098847]
  We first investigate the effects of synthetic data in synthetic-to-real novel view synthesis. We propose to introduce geometry-aware contrastive learning to learn multi-view consistent features with geometric constraints. Our method can render images with higher quality and better fine-grained details, outperforming existing generalizable novel view synthesis methods in terms of PSNR, SSIM, and LPIPS.
  arXiv  Detail & Related papers  (2023-03-20T12:06:14Z)
• LesionAid: Vision Transformers-based Skin Lesion Generation and Classification [0.0]
  This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. The framework consists of four main phases: ViTGANs, Image processing, and explainable AI.
  arXiv  Detail & Related papers  (2023-02-02T13:52:54Z)
• A Deep Learning Generative Model Approach for Image Synthesis of Plant Leaves [62.997667081978825]
  We generate via advanced Deep Learning (DL) techniques artificial leaf images in an automatized way. We aim to dispose of a source of training samples for AI applications for modern crop management.
  arXiv  Detail & Related papers  (2021-11-05T10:53:35Z)
• You Only Need Adversarial Supervision for Semantic Image Synthesis [84.83711654797342]
  We propose a novel, simplified GAN model, which needs only adversarial supervision to achieve high quality results. We show that images synthesized by our model are more diverse and follow the color and texture of real images more closely.
  arXiv  Detail & Related papers  (2020-12-08T23:00:48Z)
• Red-GAN: Attacking class imbalance via conditioned generation. Yet another perspective on medical image synthesis for skin lesion dermoscopy and brain tumor MRI [5.075029145724692]
  We propose a data augmentation protocol based on generative adversarial networks. We validate the approach on two medical datasets: BraTS, ISIC.
  arXiv  Detail & Related papers  (2020-04-22T17:38:48Z)
• Melanoma Detection using Adversarial Training and Deep Transfer Learning [6.22964000148682]
  We propose a two-stage framework for automatic classification of skin lesion images. In the first stage, we leverage the inter-class variation of the data distribution for the task of conditional image synthesis. In the second stage, we train a deep convolutional neural network for skin lesion classification.
  arXiv  Detail & Related papers  (2020-04-14T22:46:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.