Discovering Latent Graphs with GFlowNets for Diverse Conditional Image Generation

• URL: http://arxiv.org/abs/2510.22107v1
• Date: Sat, 25 Oct 2025 01:25:50 GMT
• Title: Discovering Latent Graphs with GFlowNets for Diverse Conditional Image Generation
• Abstract summary: Capturing diversity is crucial in conditional and prompt-based image generation.<n>Traditional methods often modify random seeds, making it difficult to discern meaningful differences between samples.<n>We propose Rainbow, a novel conditional image generation framework.
• Score: 16.766718482719796
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Capturing diversity is crucial in conditional and prompt-based image generation, particularly when conditions contain uncertainty that can lead to multiple plausible outputs. To generate diverse images reflecting this diversity, traditional methods often modify random seeds, making it difficult to discern meaningful differences between samples, or diversify the input prompt, which is limited in verbally interpretable diversity. We propose Rainbow, a novel conditional image generation framework, applicable to any pretrained conditional generative model, that addresses inherent condition/prompt uncertainty and generates diverse plausible images. Rainbow is based on a simple yet effective idea: decomposing the input condition into diverse latent representations, each capturing an aspect of the uncertainty and generating a distinct image. First, we integrate a latent graph, parameterized by Generative Flow Networks (GFlowNets), into the prompt representation computation. Second, leveraging GFlowNets' advanced graph sampling capabilities to capture uncertainty and output diverse trajectories over the graph, we produce multiple trajectories that collectively represent the input condition, leading to diverse condition representations and corresponding output images. Evaluations on natural image and medical image datasets demonstrate Rainbow's improvement in both diversity and fidelity across image synthesis, image generation, and counterfactual generation tasks.

Related papers

• Semantic Browsing: Controllable Diversity for Image Generation [51.503726779537]
  We introduce a method for controlled diversity that enables Semantic Browsing.<n>We exploit the fact that recent text-to-image models are trained on elaborated captions.<n>This enables a paradigm shift: instead of relying on variation within the text-to-image model, we induce diversity directly at the text level.
  arXiv  Detail & Related papers  (2026-06-22T17:59:17Z)
• Towards Generalized Multi-Image Editing for Unified Multimodal Models [56.620038824933566]
  Unified Multimodal Models (UMMs) integrate multimodal understanding and generation.<n>UMMs are limited to maintaining visual consistency and disambiguating visual cues when referencing details across multiple input images.<n>We propose a scalable multi-image editing framework for UMMs that explicitly distinguishes image identities and generalizes to variable input counts.
  arXiv  Detail & Related papers  (2026-01-09T06:42:49Z)
• A Simple Approach to Unifying Diffusion-based Conditional Generation [63.389616350290595]
  We introduce a simple, unified framework to handle diverse conditional generation tasks.<n>Our approach enables versatile capabilities via different inference-time sampling schemes.<n>Our model supports additional capabilities like non-spatially aligned and coarse conditioning.
  arXiv  Detail & Related papers  (2024-10-15T09:41:43Z)
• Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling [49.41822427811098]
  We present Kaleido, a novel approach that enhances the diversity of samples by incorporating autoregressive latent priors. Kaleido integrates an autoregressive language model that encodes the original caption and generates latent variables. We show that Kaleido adheres closely to the guidance provided by the generated latent variables, demonstrating its capability to effectively control and direct the image generation process.
  arXiv  Detail & Related papers  (2024-05-31T17:41:11Z)
• Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation [17.035117118768945]
  We propose a plug-in projected gradient descent (PGD) like method for diverse and controllable image generation. The key idea is attacking the pre-trained deterministic generative models by adding a micro perturbation to the input condition. Our work opens the door to applying adversarial attack to low-level vision tasks.
  arXiv  Detail & Related papers  (2024-03-13T06:57:23Z)
• Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation [0.0]
  We introduce Diverse Diffusion, a method for boosting image diversity beyond gender and ethnicity. Our approach contributes to the creation of more inclusive and representative AI-generated art.
  arXiv  Detail & Related papers  (2023-10-19T08:48:23Z)
• Real-World Image Variation by Aligning Diffusion Inversion Chain [53.772004619296794]
  A domain gap exists between generated images and real-world images, which poses a challenge in generating high-quality variations of real-world images. We propose a novel inference pipeline called Real-world Image Variation by ALignment (RIVAL) Our pipeline enhances the generation quality of image variations by aligning the image generation process to the source image's inversion chain.
  arXiv  Detail & Related papers  (2023-05-30T04:09:47Z)
• Auto-regressive Image Synthesis with Integrated Quantization [55.51231796778219]
  This paper presents a versatile framework for conditional image generation. It incorporates the inductive bias of CNNs and powerful sequence modeling of auto-regression. Our method achieves superior diverse image generation performance as compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-21T22:19:17Z)
• High-Quality Pluralistic Image Completion via Code Shared VQGAN [51.7805154545948]
  We present a novel framework for pluralistic image completion that can achieve both high quality and diversity at much faster inference speed. Our framework is able to learn semantically-rich discrete codes efficiently and robustly, resulting in much better image reconstruction quality.
  arXiv  Detail & Related papers  (2022-04-05T01:47:35Z)
• Spatially Multi-conditional Image Generation [80.04130168156792]
  We propose a novel neural architecture to address the problem of multi-conditional image generation. The proposed method uses a transformer-like architecture operating pixel-wise, which receives the available labels as input tokens. Our experiments on three benchmark datasets demonstrate the clear superiority of our method over the state-of-the-art and the compared baselines.
  arXiv  Detail & Related papers  (2022-03-25T17:57:13Z)
• MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning [46.4308182215488]
  A text-based image intuitively contains abundant and complex multimodal relational content. We propose the Multimodal relAtional Graph adversarIal inferenCe framework for diverse and unpaired TextCap. We validate the effectiveness of MAGIC in generating diverse captions from different relational information items of an image.
  arXiv  Detail & Related papers  (2021-12-13T11:00:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.