Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation

• URL: http://arxiv.org/abs/2310.12583v1
• Date: Thu, 19 Oct 2023 08:48:23 GMT
• Title: Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation
• Abstract summary: We introduce Diverse Diffusion, a method for boosting image diversity beyond gender and ethnicity. Our approach contributes to the creation of more inclusive and representative AI-generated art.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent diffusion models excel at producing high-quality images from text. Yet, concerns appear about the lack of diversity in the generated imagery. To tackle this, we introduce Diverse Diffusion, a method for boosting image diversity beyond gender and ethnicity, spanning into richer realms, including color diversity.Diverse Diffusion is a general unsupervised technique that can be applied to existing text-to-image models. Our approach focuses on finding vectors in the Stable Diffusion latent space that are distant from each other. We generate multiple vectors in the latent space until we find a set of vectors that meets the desired distance requirements and the required batch size.To evaluate the effectiveness of our diversity methods, we conduct experiments examining various characteristics, including color diversity, LPIPS metric, and ethnicity/gender representation in images featuring humans.The results of our experiments emphasize the significance of diversity in generating realistic and varied images, offering valuable insights for improving text-to-image models. Through the enhancement of image diversity, our approach contributes to the creation of more inclusive and representative AI-generated art.

Related papers

• Semantic Browsing: Controllable Diversity for Image Generation [51.503726779537]
  We introduce a method for controlled diversity that enables Semantic Browsing.<n>We exploit the fact that recent text-to-image models are trained on elaborated captions.<n>This enables a paradigm shift: instead of relying on variation within the text-to-image model, we induce diversity directly at the text level.
  arXiv  Detail & Related papers  (2026-06-22T17:59:17Z)
• GASS: Geometry-Aware Spherical Sampling for Disentangled Diversity Enhancement in Text-to-Image Generation [32.63174739701972]
  Despite high semantic alignment, modern text-to-image (T2I) generative models still struggle to synthesize diverse images from a given prompt.<n>We introduce Geometry-Aware Spherical Sampling (GASS) to enhance diversity by explicitly controlling both prompt-dependent and prompt-independent sources of variation.<n>Our experiments on different frozen T2I backbones and benchmarks demonstrate the effectiveness of disentangled diversity enhancement with minimal impact on image fidelity and semantic alignment.
  arXiv  Detail & Related papers  (2026-02-19T09:41:32Z)
• DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPO [50.89703227426486]
  Reinforcement learning (RL) improves image generation quality significantly by comparing the relative performance of images generated within the same group.<n>In the later stages of training, the model tends to produce homogenized outputs, lacking creativity and visual diversity.<n>This issue can be analyzed from both reward modeling and generation dynamics perspectives.
  arXiv  Detail & Related papers  (2025-12-25T05:37:37Z)
• DiverseVAR: Balancing Diversity and Quality of Next-Scale Visual Autoregressive Models [23.12099227251494]
  We introduce Diverse VAR, a framework that enhances the diversity of text-conditioned visual autoregressive models ( VAR) at test time.<n>Var models have emerged as strong competitors to diffusion and flow models for image generation.<n>Var models suffer from a critical limitation in diversity, often producing nearly identical images even for simple prompts.
  arXiv  Detail & Related papers  (2025-11-26T14:06:52Z)
• Training-Free Generation of Diverse and High-Fidelity Images via Prompt Semantic Space Optimization [50.5332987313297]
  We propose Token-Prompt embedding Space Optimization (TPSO), a training-free and model-agnostic module.<n>TPSO introduces learnable parameters to explore underrepresented regions of the token embedding space, reducing the tendency of the model to repeatedly generate samples from strong modes of the learned distribution.<n>In experiments on MS-COCO and three diffusion backbones, TPSO significantly enhances generative diversity, improving baseline performance from 1.10 to 4.18 points, without sacrificing image quality.
  arXiv  Detail & Related papers  (2025-11-25T00:42:09Z)
• Evolve to Inspire: Novelty Search for Diverse Image Generation [6.040326113136291]
  We introduce WANDER, a novelty search-based approach to generating diverse sets of images from a single input prompt.<n>We employ a Large Language Model (LLM) for semantic evolution of diverse sets of images, and using CLIP embeddings to quantify novelty.<n>We additionally apply emitters to guide the search into distinct regions of the prompt space, and demonstrate that they boost the diversity of the generated images.
  arXiv  Detail & Related papers  (2025-11-01T19:58:07Z)
• Discovering Latent Graphs with GFlowNets for Diverse Conditional Image Generation [16.766718482719796]
  Capturing diversity is crucial in conditional and prompt-based image generation.<n>Traditional methods often modify random seeds, making it difficult to discern meaningful differences between samples.<n>We propose Rainbow, a novel conditional image generation framework.
  arXiv  Detail & Related papers  (2025-10-25T01:25:50Z)
• Diffusion Autoencoders for Few-shot Image Generation in Hyperbolic Space [35.516111930052105]
  Few-shot image generation aims to generate diverse and high-quality images for an unseen class given only a few examples in that class. We propose Hyperbolic Diffusion Autoencoders (HypDAE), a novel approach that operates in hyperbolic space to capture hierarchical relationships among images and texts from seen categories.
  arXiv  Detail & Related papers  (2024-11-27T00:45:51Z)
• GRADE: Quantifying Sample Diversity in Text-to-Image Models [66.12068246962762]
  We propose GRADE: Granular Attribute Diversity Evaluation, an automatic method for quantifying sample diversity. We measure the overall diversity of 12 T2I models using 400 concept-attribute pairs, revealing that all models display limited variation. Our work proposes a modern, semantically-driven approach to measure sample diversity and highlights the stunning homogeneity in outputs by T2I models.
  arXiv  Detail & Related papers  (2024-10-29T23:10:28Z)
• Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance [12.33170407159189]
  State-of-the-art text-to-image generative models struggle to depict everyday objects with the true diversity of the real world. We introduce an inference time intervention, contextualized Vendi Score Guidance (c-VSG), that guides the backwards steps of latent diffusion models to increase the diversity of a sample. We find that c-VSG substantially increases the diversity of generated images, both for the worst performing regions and on average, while simultaneously maintaining or improving image quality and consistency.
  arXiv  Detail & Related papers  (2024-06-06T23:35:51Z)
• Diff-Mosaic: Augmenting Realistic Representations in Infrared Small Target Detection via Diffusion Prior [63.64088590653005]
  We propose Diff-Mosaic, a data augmentation method based on the diffusion model. We introduce an enhancement network called Pixel-Prior, which generates highly coordinated and realistic Mosaic images. In the second stage, we propose an image enhancement strategy named Diff-Prior. This strategy utilizes diffusion priors to model images in the real-world scene.
  arXiv  Detail & Related papers  (2024-06-02T06:23:05Z)
• Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling [49.41822427811098]
  We present Kaleido, a novel approach that enhances the diversity of samples by incorporating autoregressive latent priors. Kaleido integrates an autoregressive language model that encodes the original caption and generates latent variables. We show that Kaleido adheres closely to the guidance provided by the generated latent variables, demonstrating its capability to effectively control and direct the image generation process.
  arXiv  Detail & Related papers  (2024-05-31T17:41:11Z)
• DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation [51.24734569887687]
  We introduce a solution that allows a pretrained T2I diffusion model to learn a set of soft prompts. These prompts offer text-guided editing capabilities and additional flexibility in controlling variation and mixing between multiple distributions. We also show the adaptability of the learned prompt distribution to other tasks, such as text-to-3D.
  arXiv  Detail & Related papers  (2023-12-21T12:11:00Z)
• StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation [103.88928334431786]
  We present a novel method for generating high-quality, stylized 3D avatars. We use pre-trained image-text diffusion models for data generation and a Generative Adversarial Network (GAN)-based 3D generation network for training. Our approach demonstrates superior performance over current state-of-the-art methods in terms of visual quality and diversity of the produced avatars.
  arXiv  Detail & Related papers  (2023-05-30T13:09:21Z)
• Real-World Image Variation by Aligning Diffusion Inversion Chain [53.772004619296794]
  A domain gap exists between generated images and real-world images, which poses a challenge in generating high-quality variations of real-world images. We propose a novel inference pipeline called Real-world Image Variation by ALignment (RIVAL) Our pipeline enhances the generation quality of image variations by aligning the image generation process to the source image's inversion chain.
  arXiv  Detail & Related papers  (2023-05-30T04:09:47Z)
• Effective Data Augmentation With Diffusion Models [65.09758931804478]
  We address the lack of diversity in data augmentation with image-to-image transformations parameterized by pre-trained text-to-image diffusion models. Our method edits images to change their semantics using an off-the-shelf diffusion model, and generalizes to novel visual concepts from a few labelled examples. We evaluate our approach on few-shot image classification tasks, and on a real-world weed recognition task, and observe an improvement in accuracy in tested domains.
  arXiv  Detail & Related papers  (2023-02-07T20:42:28Z)
• Few-shot Image Generation via Masked Discrimination [20.998032566820907]
  Few-shot image generation aims to generate images of high quality and great diversity with limited data. It is difficult for modern GANs to avoid overfitting when trained on only a few images. This work presents a novel approach to realize few-shot GAN adaptation via masked discrimination.
  arXiv  Detail & Related papers  (2022-10-27T06:02:22Z)
• Random Network Distillation as a Diversity Metric for Both Image and Text Generation [62.13444904851029]
  We develop a new diversity metric that can be applied to data, both synthetic and natural, of any type. We validate and deploy this metric on both images and text.
  arXiv  Detail & Related papers  (2020-10-13T22:03:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.