coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation

• URL: http://arxiv.org/abs/2603.12829v1
• Date: Fri, 13 Mar 2026 09:32:06 GMT
• Title: coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation
• Abstract summary: We propose coDrawAgents, an interactive multi-agent dialogue framework with four specialized agents.<n>The Interpreter decides between a direct text-to-image pathway and a layout-aware multi-agent process.<n>The Planner proposes layouts for objects with the same semantic priority level while grounding decisions in the evolving visual context.<n>The Checker introduces an explicit error-correction mechanism by validating spatial consistency and attribute alignment.<n>The Painter synthesizes the image step by step, incorporating newly planned objects into the canvas to provide richer context for subsequent iterations.
• Score: 48.027946344020314
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Text-to-image generation has advanced rapidly, but existing models still struggle with faithfully composing multiple objects and preserving their attributes in complex scenes. We propose coDrawAgents, an interactive multi-agent dialogue framework with four specialized agents: Interpreter, Planner, Checker, and Painter that collaborate to improve compositional generation. The Interpreter adaptively decides between a direct text-to-image pathway and a layout-aware multi-agent process. In the layout-aware mode, it parses the prompt into attribute-rich object descriptors, ranks them by semantic salience, and groups objects with the same semantic priority level for joint generation. Guided by the Interpreter, the Planner adopts a divide-and-conquer strategy, incrementally proposing layouts for objects with the same semantic priority level while grounding decisions in the evolving visual context of the canvas. The Checker introduces an explicit error-correction mechanism by validating spatial consistency and attribute alignment, and refining layouts before they are rendered. Finally, the Painter synthesizes the image step by step, incorporating newly planned objects into the canvas to provide richer context for subsequent iterations. Together, these agents address three key challenges: reducing layout complexity, grounding planning in visual context, and enabling explicit error correction. Extensive experiments on benchmarks GenEval and DPG-Bench demonstrate that coDrawAgents substantially improves text-image alignment, spatial accuracy, and attribute binding compared to existing methods.

Related papers

• Learning to Compose: Revisiting Proxy Task Design for Zero-Shot Composed Image Retrieval [31.201448685871558]
  Composed Image Retrieval (CIR) retrieves a target image from a reference image and a textual modification.<n>We propose FoCo, which models composition as two coordinated stages: focusing on modification-relevant visual content, and then completing the target semantics.<n>Experiments on four ZS-CIR benchmarks show FoCo's state-of-the-art performance and improved generalization.
  arXiv  Detail & Related papers  (2026-07-01T03:20:06Z)
• APE: Agentic Prompt Enhancer for Image Generation and Editing [58.43341902967105]
  Agentic Prompt Enhancer (APE) is a lightweight framework that post-trains small language models (SLMs) as prompt-enhancement agents.<n>APE supports both single-agent rewriting and role-specialized multi-agent enhancement.<n>Its single-agent instantiation, SAPE, rewrites the prompt in one pass, while its multi-agent instantiation, MAPE, decomposes enhancement into a router--rewriter--composer process.
  arXiv  Detail & Related papers  (2026-05-29T17:59:31Z)
• I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing [59.434028565445885]
  I2E is a novel "Decompose-then-Action" paradigm that revisits image editing as an actionable interaction process within a structured environment.<n>I2E utilizes a Decomposer to transform unstructured images into discrete, manipulable object layers and then introduces a physics-aware Vision-Language-Action Agent to parse complex instructions.<n>I2E significantly outperforms state-of-the-art methods in handling complex compositional instructions, maintaining physical plausibility, and ensuring multi-turn editing stability.
  arXiv  Detail & Related papers  (2026-01-07T09:29:57Z)
• LayoutAgent: A Vision-Language Agent Guided Compositional Diffusion for Spatial Layout Planning [18.207887244259897]
  Designing realistic multi-object scenes requires planning spatial layouts that respect semantic relations and physical plausibility.<n>We propose LayoutAgent, an agentic framework that unifies vision-language reasoning with compositional diffusion for layout generation.<n>Our method first employs visual-language model to preprocess the inputs through segmentation, object size estimation, scene graph construction, and prompt rewriting.<n>In the end, a foreground-conditioned image generator composes the complete scene by rendering the objects into the planned layout guided by designed prompts.
  arXiv  Detail & Related papers  (2025-09-24T20:41:04Z)
• Text4Seg++: Advancing Image Segmentation via Generative Language Modeling [52.07442359419673]
  We propose a novel text-as-mask paradigm that casts image segmentation as a text generation problem.<n>Key innovation is semantic descriptors, a new textual representation of segmentation masks.<n>Experiments on natural and remote sensing datasets show that Text4Seg++ consistently outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2025-09-08T04:07:14Z)
• CAL-RAG: Retrieval-Augmented Multi-Agent Generation for Content-Aware Layout Design [6.830055289299306]
  CAL-RAG is a retrieval-augmented, agentic framework for content-aware layout generation.<n>We implement our framework using LangGraph and evaluate it on a benchmark rich in semantic variability.<n>Results demonstrate that combining retrieval augmentation with agentic multi-step reasoning yields a scalable, interpretable, and high-fidelity solution.
  arXiv  Detail & Related papers  (2025-06-27T06:09:56Z)
• VSC: Visual Search Compositional Text-to-Image Diffusion Model [15.682990658945682]
  We introduce a novel compositional generation method that leverages pairwise image embeddings to improve attribute-object binding.<n>Our approach decomposes complex prompts into sub-prompts, generates corresponding images, and computes visual prototypes that fuse with text embeddings to enhance representation.<n>Our approaches outperform existing compositional text-to-image diffusion models on the benchmark T2I CompBench, achieving better image quality, evaluated by humans, and emerging robustness under scaling number of binding pairs in the prompt.
  arXiv  Detail & Related papers  (2025-05-02T08:31:43Z)
• Vision-Aware Text Features in Referring Image Segmentation: From Object Understanding to Context Understanding [26.768147543628096]
  We propose a novel framework that emphasizes object and context comprehension inspired by human cognitive processes. Our method achieves significant performance improvements on three benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-12T16:38:48Z)
• Divide and Conquer: Language Models can Plan and Self-Correct for Compositional Text-to-Image Generation [72.6168579583414]
  CompAgent is a training-free approach for compositional text-to-image generation with a large language model (LLM) agent as its core. Our approach achieves more than 10% improvement on T2I-CompBench, a comprehensive benchmark for open-world compositional T2I generation.
  arXiv  Detail & Related papers  (2024-01-28T16:18:39Z)
• LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts [60.54912319612113]
  Diffusion-based generative models have significantly advanced text-to-image generation but encounter challenges when processing lengthy and intricate text prompts. We present a novel approach leveraging Large Language Models (LLMs) to extract critical components from text prompts. Our evaluation on complex prompts featuring multiple objects demonstrates a substantial improvement in recall compared to baseline diffusion models.
  arXiv  Detail & Related papers  (2023-10-16T17:57:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.