Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance

• URL: http://arxiv.org/abs/2606.27371v1
• Date: Thu, 25 Jun 2026 17:59:47 GMT
• Title: Don't Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance
• Abstract summary: State-of-the-art flow models generate stunning images from text or image prompts.<n>They suffer from diversity collapse when generating multiple samples under the same conditioning.<n>We introduce an efficient, training-free self-guidance mechanism to mitigate diversity collapse without requiring additional reward models.
• Score: 26.478519452337434
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple samples under the same conditioning. Existing methods address this issue via either latent guidance, which has limited effectiveness, or sample selection, which relies on external reward models that incur significant inference-time overhead. In this work, we introduce an efficient, training-free self-guidance mechanism to mitigate diversity collapse without requiring additional reward models. Specifically, we disperse the internal features of the flow model during batch generation with feature self-guidance. Further, to keep the features close to the manifold, we introduce a manifold regularization step that projects these dispersed features back onto the data manifold, ensuring diverse generation without sacrificing alignment with the input conditions. Our method integrates seamlessly as a plug-and-play module into pretrained flow models, adding only a marginal inference cost. Experiments demonstrate significant improvements in diversity while preserving fidelity across several conditional flow models, including multi-step and few-step text-to-image, depth-to-image, and reference image generation.

Related papers

• Training-Free Self-Correction for Multimodal Masked Diffusion Models [61.84305395626145]
  We propose a training-free self-correction framework that exploits the inductive biases of pre-trained masked diffusion models.<n>Our method significantly improves generation quality on text-to-image generation and multimodal understanding tasks with reduced sampling steps.
  arXiv  Detail & Related papers  (2026-02-02T23:58:15Z)
• Training-Free Generation of Diverse and High-Fidelity Images via Prompt Semantic Space Optimization [50.5332987313297]
  We propose Token-Prompt embedding Space Optimization (TPSO), a training-free and model-agnostic module.<n>TPSO introduces learnable parameters to explore underrepresented regions of the token embedding space, reducing the tendency of the model to repeatedly generate samples from strong modes of the learned distribution.<n>In experiments on MS-COCO and three diffusion backbones, TPSO significantly enhances generative diversity, improving baseline performance from 1.10 to 4.18 points, without sacrificing image quality.
  arXiv  Detail & Related papers  (2025-11-25T00:42:09Z)
• OSCAR: Orthogonal Stochastic Control for Alignment-Respecting Diversity in Flow Matching [14.664226708184676]
  Flow-based text-to-image models follow deterministic trajectories, forcing users to repeatedly sample to discover diverse modes.<n>We present a training-free, inference-time control mechanism that makes the flow itself diversity-aware.
  arXiv  Detail & Related papers  (2025-10-10T07:07:19Z)
• Image Generation Diversity Issues and How to Tame Them [8.858030256056095]
  Generative methods now produce outputs nearly indistinguishable from real data but often fail to fully capture the data distribution.<n>In this paper, we draw attention to the current lack of diversity in generative models and the inability of common metrics to measure this.<n>We achieve this by framing diversity as an image retrieval problem, where we measure how many real images can be retrieved using synthetic data as queries.
  arXiv  Detail & Related papers  (2024-11-25T08:00:21Z)
• A Simple Approach to Unifying Diffusion-based Conditional Generation [63.389616350290595]
  We introduce a simple, unified framework to handle diverse conditional generation tasks.<n>Our approach enables versatile capabilities via different inference-time sampling schemes.<n>Our model supports additional capabilities like non-spatially aligned and coarse conditioning.
  arXiv  Detail & Related papers  (2024-10-15T09:41:43Z)
• Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling [49.41822427811098]
  We present Kaleido, a novel approach that enhances the diversity of samples by incorporating autoregressive latent priors. Kaleido integrates an autoregressive language model that encodes the original caption and generates latent variables. We show that Kaleido adheres closely to the guidance provided by the generated latent variables, demonstrating its capability to effectively control and direct the image generation process.
  arXiv  Detail & Related papers  (2024-05-31T17:41:11Z)
• Ensembling Diffusion Models via Adaptive Feature Aggregation [14.663257610094625]
  Leveraging multiple high-quality models to produce stronger generation ability is valuable, but has not been extensively studied.<n>Existing methods primarily adopt parameter merging strategies to produce a new static model.<n>We propose Adaptive Feature Aggregation (AFA), which dynamically adjusts the contributions of multiple models at the feature level according to various states.
  arXiv  Detail & Related papers  (2024-05-27T11:55:35Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• DiffBlender: Composable and Versatile Multimodal Text-to-Image Diffusion Models [20.281651578265624]
  We propose a multimodal T2I diffusion model, which is capable of processing all three modalities within a single architecture.<n>We demonstrate that DiffBlender effectively integrates multiple sources of information and supports diverse applications in detailed image synthesis.
  arXiv  Detail & Related papers  (2023-05-24T14:31:20Z)
• Deep Equilibrium Approaches to Diffusion Models [1.4275201654498746]
  Diffusion-based generative models are extremely effective in generating high-quality images. These models typically require long sampling chains to produce high-fidelity images. We look at diffusion models through a different perspective, that of a (deep) equilibrium (DEQ) fixed point model.
  arXiv  Detail & Related papers  (2022-10-23T22:02:19Z)
• Auto-regressive Image Synthesis with Integrated Quantization [55.51231796778219]
  This paper presents a versatile framework for conditional image generation. It incorporates the inductive bias of CNNs and powerful sequence modeling of auto-regression. Our method achieves superior diverse image generation performance as compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-21T22:19:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.