Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning

• URL: http://arxiv.org/abs/2604.04746v3
• Date: Wed, 08 Apr 2026 01:34:51 GMT
• Title: Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning
• Abstract summary: We introduce process-driven image generation, a multi-step paradigm that decomposes synthesis into an interleaved reasoning trajectory.<n>A core challenge of process-driven generation stems from the ambiguity of intermediate states.<n>We address this through dense, step-wise supervision that maintains two complementary constraints.
• Score: 59.262311672150055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Humans paint images incrementally: they plan a global layout, sketch a coarse draft, inspect, and refine details, and most importantly, each step is grounded in the evolving visual states. However, can unified multimodal models trained on text-image interleaved datasets also imagine the chain of intermediate states? In this paper, we introduce process-driven image generation, a multi-step paradigm that decomposes synthesis into an interleaved reasoning trajectory of thoughts and actions. Rather than generating images in a single step, our approach unfolds across multiple iterations, each consisting of 4 stages: textual planning, visual drafting, textual reflection, and visual refinement. The textual reasoning explicitly conditions how the visual state should evolve, while the generated visual intermediate in turn constrains and grounds the next round of textual reasoning. A core challenge of process-driven generation stems from the ambiguity of intermediate states: how can models evaluate each partially-complete image? We address this through dense, step-wise supervision that maintains two complementary constraints: for the visual intermediate states, we enforce the spatial and semantic consistency; for the textual intermediate states, we preserve the prior visual knowledge while enabling the model to identify and correct prompt-violating elements. This makes the generation process explicit, interpretable, and directly supervisable. To validate proposed method, we conduct experiments under various text-to-image generation benchmarks.

Related papers

• FlowInOne:Unifying Multimodal Generation as Image-in, Image-out Flow Matching [86.31254356971506]
  FlowInOne is a framework that reformulates multimodal generation as a purely visual flow.<n>It unifies text-to-image generation, layout-guided editing, and visual instruction following under one coherent paradigm.<n>It achieves state-of-the-art performance across all unified generation tasks, surpassing both open-source models and competitive commercial systems.
  arXiv  Detail & Related papers  (2026-04-08T07:22:08Z)
• All-in-One Conditioning for Text-to-Image Synthesis [45.22434803596108]
  We propose a novel approach that grounds text-to-image synthesis within the framework of scene graph structures.<n>We introduce a zero-shot, scene graph-based conditioning mechanism that generates soft visual guidance during inference.<n>This enables the model to maintain text-image alignment while supporting lightweight, coherent, and diverse image synthesis.
  arXiv  Detail & Related papers  (2026-02-09T20:16:19Z)
• UniModel: A Visual-Only Framework for Unified Multimodal Understanding and Generation [51.31795451147935]
  We present a unified generative model that supports visual understanding and visual generation within a single pixel-to-pixel diffusion framework.<n>Our goal is to achieve unification along three axes: the model, the tasks, and the representations.<n> Experiments on text-to-image synthesis and image-to-text understanding demonstrate strong cross-modal alignment.
  arXiv  Detail & Related papers  (2025-11-21T03:02:10Z)
• Contextualized Diffusion Models for Text-Guided Image and Video Generation [67.69171154637172]
  Conditional diffusion models have exhibited superior performance in high-fidelity text-guided visual generation and editing. We propose a novel and general contextualized diffusion model (ContextDiff) by incorporating the cross-modal context encompassing interactions and alignments between text condition and visual sample. We generalize our model to both DDPMs and DDIMs with theoretical derivations, and demonstrate the effectiveness of our model in evaluations with two challenging tasks: text-to-image generation, and text-to-video editing.
  arXiv  Detail & Related papers  (2024-02-26T15:01:16Z)
• Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation [11.80682025950519]
  In this work, we investigate the diffusion (physics) in diffusion (machine learning) properties. We propose our Cyclic One-Way Diffusion (COW) method to control the direction of diffusion phenomenon. Our method provides a novel perspective to understand the task needs and is applicable to a wider range of customization scenarios.
  arXiv  Detail & Related papers  (2023-06-14T05:25:06Z)
• HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation [73.3790833537313]
  Controllable person image generation promotes a wide range of applications such as digital human interaction and virtual try-on. We propose HumanDiffusion, a coarse-to-fine alignment diffusion framework, for text-driven person image generation.
  arXiv  Detail & Related papers  (2022-11-11T14:30:34Z)
• Improving Generation and Evaluation of Visual Stories via Semantic Consistency [72.00815192668193]
  Given a series of natural language captions, an agent must generate a sequence of images that correspond to the captions. Prior work has introduced recurrent generative models which outperform synthesis text-to-image models on this task. We present a number of improvements to prior modeling approaches, including the addition of a dual learning framework.
  arXiv  Detail & Related papers  (2021-05-20T20:42:42Z)
• Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze [6.6358421117698665]
  We take as our starting point a state-of-the-art image captioning system. We develop several model variants that exploit information from human gaze patterns recorded during language production. Our experiments and analyses confirm that better descriptions can be obtained by exploiting gaze-driven attention.
  arXiv  Detail & Related papers  (2020-11-09T17:45:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.