Initialization is Half the Battle: Generating Diverse Images from a Guidance Potential Posterior

• URL: http://arxiv.org/abs/2606.02453v1
• Date: Mon, 01 Jun 2026 16:25:10 GMT
• Title: Initialization is Half the Battle: Generating Diverse Images from a Guidance Potential Posterior
• Abstract summary: We formulate selecting the initial noise from a guidance potential posterior, which effectively re-weights the prior towards diversity-rich regions.<n>To sample from this distribution efficiently, we introduce Diversity-inducing Initialization (DivIn)<n>Our method serves as an inference-time diversity enhancement compatible with both diffusion and flow matching models.
• Score: 51.62608628984139
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Despite the remarkable fidelity of generative models, they frequently suffer from mode collapse. Existing strategies for enhancing diversity predominantly focus on intervening during the generation trajectory. We identify a critical oversight that the standard Gaussian initialization often causes trajectories to collapse into dominant modes because it is agnostic to the guidance potential landscape. In this work, we formulate selecting the initial noise from a guidance potential posterior, which effectively re-weights the prior towards diversity-rich regions. To sample from this distribution efficiently, we introduce Diversity-inducing Initialization (DivIn), which leverages Langevin dynamics to actively navigate the initialization landscape, steering initial noise away from collapsing regions while anchoring them to the valid data manifold. Our method serves as an inference-time diversity enhancement compatible with both diffusion and flow matching models. Extensive experiments show that DivIn exhibits a superior performance in both class-to-image and text-to-image scenarios. Furthermore, we highlight that as DivIn is orthogonal to trajectory-based methods, combining them significantly expands the diversity-quality Pareto frontier beyond what either achieves in isolation.

Related papers

• Exploring More to Solve More: Boosting Diversity in Text Diffusion Models via Entropy-Based Guidance [29.822116353065386]
  We introduce a training-free Semantic-Aware Kernel Entropy guidance method.<n>Our method computes the order-2 Rényi entropy over a kernel Gram matrix.<n>By linearizing this objective in the embedding space, we derive a tractable guidance signal.
  arXiv  Detail & Related papers  (2026-07-10T07:15:11Z)
• BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception [54.31138496553705]
  We propose Bayesian Visual Search, a framework that formulates perception as a global optimization problem over a continuous spatial-scale manifold.<n>We provide theoretical guarantees via sub-linear bounds regret, and extensive experiments demonstrate that BVS significantly outperforms state-of-the-art baselines with a superior trade-off between accuracy and efficiency.
  arXiv  Detail & Related papers  (2026-07-03T10:42:34Z)
• Anomaly-Preference Image Generation [41.96735569500412]
  We introduce Anomaly Preference Optimization, a novel paradigm that reformulates anomaly generation as a preference learning problem.<n>We show that our approach significantly outperforms existing baselines,achieving state-of-the-art performance in both realism and diversity.
  arXiv  Detail & Related papers  (2026-05-04T10:37:09Z)
• Scaling Dense Event-Stream Pretraining from Visual Foundation Models [112.44243079477137]
  We launch a novel self-supervised pretraining method that distills visual foundation models (VFMs) to push the boundaries of event representation at scale.<n>We curate an extensive synchronized image-event collection to amplify cross-modal alignment.<n>We extend the alignment objective to semantic structures provided off-the-shelf by VFMs, indicating a broader receptive field and stronger supervision.
  arXiv  Detail & Related papers  (2026-03-04T12:06:09Z)
• Training-Free Generation of Diverse and High-Fidelity Images via Prompt Semantic Space Optimization [50.5332987313297]
  We propose Token-Prompt embedding Space Optimization (TPSO), a training-free and model-agnostic module.<n>TPSO introduces learnable parameters to explore underrepresented regions of the token embedding space, reducing the tendency of the model to repeatedly generate samples from strong modes of the learned distribution.<n>In experiments on MS-COCO and three diffusion backbones, TPSO significantly enhances generative diversity, improving baseline performance from 1.10 to 4.18 points, without sacrificing image quality.
  arXiv  Detail & Related papers  (2025-11-25T00:42:09Z)
• Enhancing Adversarial Transferability by Balancing Exploration and Exploitation with Gradient-Guided Sampling [82.52485740425321]
  Adrial attacks present a critical challenge to deep neural networks' robustness.<n> transferability of adversarial attacks faces a dilemma between Exploitation (maximizing attack potency) and Exploration (enhancing cross-model generalization)
  arXiv  Detail & Related papers  (2025-11-01T05:43:47Z)
• Diverse Text-to-Image Generation via Contrastive Noise Optimization [60.48914865049489]
  Text-to-image (T2I) diffusion models have demonstrated impressive performance in generating high-fidelity images.<n>Existing approaches typically optimize intermediate latents or text conditions during inference.<n>We introduce Contrastive Noise Optimization, a simple yet effective method that addresses the diversity issue from a distinct perspective.
  arXiv  Detail & Related papers  (2025-10-04T13:51:32Z)
• Stage-wise Dynamics of Classifier-Free Guidance in Diffusion Models [13.030934039187171]
  CFG is widely used to improve conditional fidelity in diffusion models, but its impact on sampling dynamics remains poorly understood.<n>We analyze CFG under multimodal conditionals and show that the sampling process unfolds in three successive stages.<n> Experiments support these predictions, showing that early strong guidance erodes global diversity, while late strong guidance suppresses fine-grained variation.
  arXiv  Detail & Related papers  (2025-09-26T07:45:20Z)
• TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Driving [20.679370777762987]
  We propose TransDiffuser, an encoder-decoder based generative trajectory planning model.<n>We exploit a simple yet effective multi-modal representation decorrelation optimization mechanism during the denoising process.<n>TransDiffuser achieves the PDMS of 94.85 on the closed-loop planning-oriented benchmark NAVSIM.
  arXiv  Detail & Related papers  (2025-05-14T12:10:41Z)
• Spontaneous Symmetry Breaking in Generative Diffusion Models [6.4322891559626125]
  Generative diffusion models have recently emerged as a leading approach for generating high-dimensional data. We show that the dynamics of these models exhibit a spontaneous symmetry breaking that divides the generative dynamics into two distinct phases. We propose a new way to understand the generative dynamics of diffusion models that has the potential to bring about higher performance and less biased fast-samplers.
  arXiv  Detail & Related papers  (2023-05-31T09:36:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.