Representation-Conditioned Diffusion Models for Guided Training Data Generation

• URL: http://arxiv.org/abs/2605.27495v1
• Date: Tue, 26 May 2026 17:32:50 GMT
• Title: Representation-Conditioned Diffusion Models for Guided Training Data Generation
• Abstract summary: We evaluate the classification performance of models trained on synthetic image datasets produced by generative deep learning.<n>We use latent diffusion models conditioned on learned representations from DINOv2, DINOv3, and CLIP.<n>Our results demonstrate that this representation-conditioned formulation significantly outperforms class-conditioned generation by a large margin.
• Score: 1.9517610560768623
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data availability remains a critical bottleneck in many deep learning applications. Large-scale datasets are often expensive to collect, curate and annotate, which can limit the scalability and applicability of supervised learning methods. In this work, we evaluate the classification performance of models trained on synthetic image datasets produced by generative deep learning. In particular, we use latent diffusion models conditioned on learned representations from DINOv2, DINOv3, and CLIP. Our results demonstrates that this representation-conditioned formulation significantly outperforms class-conditioned generation by a large margin (+10.76 p.p. top-1 accuracy on ImageNet100), by improving sample quality and mode coverage. Furthermore, by scaling the size of the synthetic dataset, we are able to outperform a classifier trained on the real data (+2.0 p.p top-1 accuracy). We also demonstrate how generated images can be used for augmentation purposes, outperforming classical augmentation methods, and how the conditioning space can be used for sample filtering to further improve training value. Collectively, these findings highlight that representation-conditioned diffusion models provide a promising approach for augmenting, complementing, or potentially replacing real-world datasets in large-scale visual learning tasks.

Related papers

• Exploring Training and Inference Scaling Laws in Generative Retrieval [50.82554729023865]
  Generative retrieval reformulates retrieval as an autoregressive generation task, where large language models generate target documents directly from a query.<n>We systematically investigate training and inference scaling laws in generative retrieval, exploring how model size, training data scale, and inference-time compute jointly influence performance.
  arXiv  Detail & Related papers  (2025-03-24T17:59:03Z)
• DreamMask: Boosting Open-vocabulary Panoptic Segmentation with Synthetic Data [61.62554324594797]
  We propose DreamMask, which explores how to generate training data in the open-vocabulary setting, and how to train the model with both real and synthetic data.<n>In general, DreamMask significantly simplifies the collection of large-scale training data, serving as a plug-and-play enhancement for existing methods.<n>For instance, when trained on COCO and tested on ADE20K, the model equipped with DreamMask outperforms the previous state-of-the-art by a substantial margin of 2.1% mIoU.
  arXiv  Detail & Related papers  (2025-01-03T19:00:00Z)
• FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration [66.61201445650323]
  Existing methods suffer from a generalization bottleneck in real-world scenarios.<n>We contribute a million-scale dataset with two notable advantages over existing training data.<n>We propose a robust model, FoundIR, to better address a broader range of restoration tasks in real-world scenarios.
  arXiv  Detail & Related papers  (2024-12-02T12:08:40Z)
• Feature-to-Image Data Augmentation: Improving Model Feature Extraction with Cluster-Guided Synthetic Samples [4.041834517339835]
  This study introduces FICAug, a novel feature-to-image data augmentation framework.<n>It is designed to improve model generalization under limited data conditions by generating structured synthetic samples.<n> Experimental results demonstrate that FICAug significantly improves classification accuracy.
  arXiv  Detail & Related papers  (2024-09-26T09:51:08Z)
• Generative Expansion of Small Datasets: An Expansive Graph Approach [13.053285552524052]
  We introduce an Expansive Synthesis model generating large-scale, information-rich datasets from minimal samples. An autoencoder with self-attention layers and optimal transport refines distributional consistency. Results show comparable performance, demonstrating the model's potential to augment training data effectively.
  arXiv  Detail & Related papers  (2024-06-25T02:59:02Z)
• DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception [78.26734070960886]
  Current perceptive models heavily depend on resource-intensive datasets. We introduce perception-aware loss (P.A. loss) through segmentation, improving both quality and controllability. Our method customizes data augmentation by extracting and utilizing perception-aware attribute (P.A. Attr) during generation.
  arXiv  Detail & Related papers  (2024-03-20T04:58:03Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Phased Data Augmentation for Training a Likelihood-Based Generative Model with Limited Data [0.0]
  Generative models excel in creating realistic images, yet their dependency on extensive datasets for training presents significant challenges. Current data-efficient methods largely focus on GAN architectures, leaving a gap in training other types of generative models. "phased data augmentation" is a novel technique that addresses this gap by optimizing training in limited data scenarios without altering the inherent data distribution.
  arXiv  Detail & Related papers  (2023-05-22T03:38:59Z)
• Instance-Conditioned GAN Data Augmentation for Representation Learning [29.36473147430433]
  We introduce DA_IC-GAN, a learnable data augmentation module that can be used off-the-shelf in conjunction with most state-of-the-art training recipes. We show that DA_IC-GAN can boost accuracy to between 1%p and 2%p with the highest capacity models. We additionally couple DA_IC-GAN with a self-supervised training recipe and show that we can also achieve an improvement of 1%p in accuracy in some settings.
  arXiv  Detail & Related papers  (2023-03-16T22:45:43Z)
• Negative Data Augmentation [127.28042046152954]
  We show that negative data augmentation samples provide information on the support of the data distribution. We introduce a new GAN training objective where we use NDA as an additional source of synthetic data for the discriminator. Empirically, models trained with our method achieve improved conditional/unconditional image generation along with improved anomaly detection capabilities.
  arXiv  Detail & Related papers  (2021-02-09T20:28:35Z)
• Improving the Performance of Fine-Grain Image Classifiers via Generative Data Augmentation [0.5161531917413706]
  We develop Data Augmentation from Proficient Pre-Training of Robust Generative Adrial Networks (DAPPER GAN) DAPPER GAN is an ML analytics support tool that automatically generates novel views of training images. We experimentally evaluate this technique on the Stanford Cars dataset, demonstrating improved vehicle make and model classification accuracy.
  arXiv  Detail & Related papers  (2020-08-12T15:29:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.