Attributed Synthetic Data Generation for Zero-shot Domain-specific Image Classification

• URL: http://arxiv.org/abs/2504.04510v1
• Date: Sun, 06 Apr 2025 14:54:10 GMT
• Title: Attributed Synthetic Data Generation for Zero-shot Domain-specific Image Classification
• Authors: Shijian Wang, Linxin Song, Ryotaro Shimizu, Masayuki Goto, Hanqian Wu,
• Abstract summary: We propose AttrSyn, which leverages large language models to generate attributed prompts.<n>Experiments for zero-shot domain-specific image classification on two fine-grained datasets show that training with synthetic images generated by AttrSyn significantly outperforms CLIP's zero-shot classification.
• Score: 3.0105723746073
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Zero-shot domain-specific image classification is challenging in classifying real images without ground-truth in-domain training examples. Recent research involved knowledge from texts with a text-to-image model to generate in-domain training images in zero-shot scenarios. However, existing methods heavily rely on simple prompt strategies, limiting the diversity of synthetic training images, thus leading to inferior performance compared to real images. In this paper, we propose AttrSyn, which leverages large language models to generate attributed prompts. These prompts allow for the generation of more diverse attributed synthetic images. Experiments for zero-shot domain-specific image classification on two fine-grained datasets show that training with synthetic images generated by AttrSyn significantly outperforms CLIP's zero-shot classification under most situations and consistently surpasses simple prompt strategies.

Related papers

• Grounding Descriptions in Images informs Zero-Shot Visual Recognition [47.66166611138081]
  We propose GRAIN, a new pretraining strategy aimed at aligning representations at both fine and coarse levels simultaneously.<n>We demonstrate the enhanced zero-shot performance of our model compared to current state-of-the art methods.
  arXiv  Detail & Related papers  (2024-12-05T18:52:00Z)
• Scaling Laws of Synthetic Images for Model Training ... for Now [54.43596959598466]
  We study the scaling laws of synthetic images generated by state of the art text-to-image models. We observe that synthetic images demonstrate a scaling trend similar to, but slightly less effective than, real images in CLIP training.
  arXiv  Detail & Related papers  (2023-12-07T18:59:59Z)
• Diversified in-domain synthesis with efficient fine-tuning for few-shot classification [64.86872227580866]
  Few-shot image classification aims to learn an image classifier using only a small set of labeled examples per class. We propose DISEF, a novel approach which addresses the generalization challenge in few-shot learning using synthetic data. We validate our method in ten different benchmarks, consistently outperforming baselines and establishing a new state-of-the-art for few-shot classification.
  arXiv  Detail & Related papers  (2023-12-05T17:18:09Z)
• Diversify, Don't Fine-Tune: Scaling Up Visual Recognition Training with Synthetic Images [35.195284384050325]
  We present a new framework leveraging off-the-shelf generative models to generate synthetic training images. We address class name ambiguity, lack of diversity in naive prompts, and domain shifts. Our framework consistently enhances recognition model performance with more synthetic data.
  arXiv  Detail & Related papers  (2023-12-04T18:35:27Z)
• ITI-GEN: Inclusive Text-to-Image Generation [56.72212367905351]
  This study investigates inclusive text-to-image generative models that generate images based on human-written prompts. We show that, for some attributes, images can represent concepts more expressively than text. We propose a novel approach, ITI-GEN, that leverages readily available reference images for Inclusive Text-to-Image GENeration.
  arXiv  Detail & Related papers  (2023-09-11T15:54:30Z)
• Image Captions are Natural Prompts for Text-to-Image Models [70.30915140413383]
  We analyze the relationship between the training effect of synthetic data and the synthetic data distribution induced by prompts. We propose a simple yet effective method that prompts text-to-image generative models to synthesize more informative and diverse training data. Our method significantly improves the performance of models trained on synthetic training data.
  arXiv  Detail & Related papers  (2023-07-17T14:38:11Z)
• StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners [58.941838860425754]
  We show that training self-supervised methods on synthetic images can match or beat the real image counterpart. We develop a multi-positive contrastive learning method, which we call StableRep. With solely synthetic images, the representations learned by StableRep surpass the performance of representations learned by SimCLR and CLIP.
  arXiv  Detail & Related papers  (2023-06-01T17:59:51Z)
• Learning Disentangled Prompts for Compositional Image Synthesis [27.99470176603746]
  We study the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images. We propose a novel source class distilled visual prompt that learns disentangled prompts of semantic (e.g., class) and domain (e.g., style) from a few images.
  arXiv  Detail & Related papers  (2023-06-01T14:56:37Z)
• Cap2Aug: Caption guided Image to Image data Augmentation [41.53127698828463]
  Cap2Aug is an image-to-image diffusion model-based data augmentation strategy using image captions as text prompts. We generate captions from the limited training images and using these captions edit the training images using an image-to-image stable diffusion model. This strategy generates augmented versions of images similar to the training images yet provides semantic diversity across the samples.
  arXiv  Detail & Related papers  (2022-12-11T04:37:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.