Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models

• URL: http://arxiv.org/abs/2609.08282v1
• Date: Tue, 08 Sep 2026 05:52:53 GMT
• Title: Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models
• Abstract summary: Unified multimodal models integrate visual understanding and generation within a single network, yet the two capabilities are commonly optimized as separate tasks.<n>We introduce Generative Grounding Feedback, a self-evolving post-training framework that uses only text prompts and the model's own visual experience.
• Score: 55.231245660246934
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Unified multimodal models integrate visual understanding and generation within a single network, yet the two capabilities are commonly optimized as separate tasks. We introduce Generative Grounding Feedback(GGF), a self-evolving post-training framework that uses only text prompts and the model's own visual experience. Given a prompt, the model first generates a visual ``dream.'' Flow-level feedback compares text-, image-, and repair-conditioned predictions at the same noisy latent state, transferring image-grounded generation directions to the prompt condition. Dream replay grounding replays this dream through captioning and re-imagination, training claim-level evidence to remain consistent across the replay while separating unrelated visual experiences. Jointly optimized, these two directions let generation provide visual grounding for understanding and understanding refine subsequent generation without paired image--text supervision. Experiments across unified models with different understanding--generation integration designs show consistent improvements in text-to-image generation together with modest gains in visual understanding.

Related papers

• Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models [34.560908829375585]
  We propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step.<n>Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts.<n>We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency.<n>This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.
  arXiv  Detail & Related papers  (2026-05-15T09:48:46Z)
• FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching [86.31254356971506]
  FlowInOne is a framework that reformulates multimodal generation as a purely visual flow.<n>It unifies text-to-image generation, layout-guided editing, and visual instruction following under one coherent paradigm.<n>It achieves state-of-the-art performance across all unified generation tasks, surpassing both open-source models and competitive commercial systems.
  arXiv  Detail & Related papers  (2026-04-08T07:22:08Z)
• Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Models [50.87835332136393]
  Chain-of-Thought (CoT) has largely improved the generation ability of unified models.<n>In this paper, we introduce visual context consistency into the reasoning of unified models.<n>We use supervised finetuning to teach the model how to plan the visual checking, conduct self-reflection and self-refinement, and use flow-GRPO to further enhance the visual consistency.
  arXiv  Detail & Related papers  (2025-12-22T18:59:03Z)
• RePrompt: Reasoning-Augmented Reprompting for Text-to-Image Generation via Reinforcement Learning [88.14234949860105]
  RePrompt is a novel reprompting framework that introduces explicit reasoning into the prompt enhancement process via reinforcement learning.<n>Our approach enables end-to-end training without human-annotated data.
  arXiv  Detail & Related papers  (2025-05-23T06:44:26Z)
• Unified Autoregressive Visual Generation and Understanding with Continuous Tokens [52.21981295470491]
  We present UniFluid, a unified autoregressive framework for joint visual generation and understanding.<n>Our unified autoregressive architecture processes multimodal image and text inputs, generating discrete tokens for text and continuous tokens for image.<n>We find though there is an inherent trade-off between the image generation and understanding task, a carefully tuned training recipe enables them to improve each other.
  arXiv  Detail & Related papers  (2025-03-17T17:58:30Z)
• VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model [38.61292051733335]
  We present VARGPT, a novel multimodal large language model (MLLM) that unifies visual understanding and generation within a single autoregressive framework.<n>VarGPT employs a next-token prediction paradigm for visual understanding and a next-scale prediction paradigm for visual autoregressive generation.<n> Notably, VARGPT naturally supports capabilities in autoregressive visual generation and instruction-to-image synthesis, showcasing its versatility in both visual understanding and generation tasks.
  arXiv  Detail & Related papers  (2025-01-21T17:50:43Z)
• Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning [2.401993998791928]
  We propose a framework that trains a lightweight vision-language mapping network to connect modalities. We introduce a multimodal contrastive objective that also improves visual relevance and story informativeness.
  arXiv  Detail & Related papers  (2024-08-12T16:15:32Z)
• Contextualized Diffusion Models for Text-Guided Image and Video Generation [67.69171154637172]
  Conditional diffusion models have exhibited superior performance in high-fidelity text-guided visual generation and editing. We propose a novel and general contextualized diffusion model (ContextDiff) by incorporating the cross-modal context encompassing interactions and alignments between text condition and visual sample. We generalize our model to both DDPMs and DDIMs with theoretical derivations, and demonstrate the effectiveness of our model in evaluations with two challenging tasks: text-to-image generation, and text-to-video editing.
  arXiv  Detail & Related papers  (2024-02-26T15:01:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.