Mise-en-Scène: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creation

• URL: http://arxiv.org/abs/2608.19000v1
• Date: Wed, 19 Aug 2026 15:02:12 GMT
• Title: Mise-en-Scène: Implicit Layout Emergence in Diffusion Transformers for Human-AI Design Co-Creation
• Abstract summary: We present Mise-en-Scne, a two-stage framework for graphic design synthesis.<n>In the first stage, a diffusion transformer adapted with a small, knockout-selected LoRA drafts a complete design in which the arrangement of the elements emerges jointly with the rendered canvas.<n>In the second stage, a deterministic match-and-place step moves the original high-resolution assets to the drafted positions, which guarantees exact asset fidelity.
• Score: 15.30671862055459
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automating graphic design synthesis from user-provided elements requires both a coherent overall composition and the exact preservation of each asset. Existing methods predict a layout as explicit bounding-box coordinates with a language model and then paste the assets into it, which separates spatial planning from visual synthesis and tends to produce rigid, mis-scaled compositions. We instead ask whether the layout can emerge implicitly inside a pretrained image-editing diffusion transformer. We present Mise-en-Scène, a two-stage framework. In the first stage, a diffusion transformer adapted with a small, knockout-selected LoRA drafts a complete design in which the arrangement of the elements emerges jointly with the rendered canvas. In the second stage, a deterministic match-and-place step moves the original high-resolution assets to the drafted positions, which guarantees exact asset fidelity and yields an editable, layered design that a designer can keep refining rather than a flat image. Notably, a minimal adaptation of the pretrained transformer already suffices, without the specialized conditioning machinery commonly introduced for multi-element generation. On the large-scale PrismLayersPlus benchmark, the designs produced by Mise-en-Scène are the closest to the ground truth in perceived quality among all compared methods, by a wide margin over both an LLM layout planner and a specialized layout transformer, while our match-and-place stage bridges the remaining fidelity gap to the ground-truth composites.

Related papers

• Resolving Representation Ambiguity in Feedforward Novel View Synthesis Transformer via Semantic-Spatial Decoupling [117.72367737965048]
  Transformer-based models have advanced feedforward novel view synthesis (NVS)<n>Current architectures mix semantic information (e.g., RGB) and spatial information (e.g., Plcker rays) into a shared feature space.<n>We propose to decouple the representation of feedforward NVS transformers into separate semantic and spatial tokens.
  arXiv  Detail & Related papers  (2026-05-18T16:09:52Z)
• CalliMaster: Mastering Page-level Chinese Calligraphy via Layout-guided Spatial Planning [37.872480365351485]
  Page-level calligraphy synthesis requires balancing glyph precision with layout composition.<n>We present textbfCalliMaster, a unified framework for controllable generation and editing.
  arXiv  Detail & Related papers  (2026-03-12T22:01:19Z)
• MagicQuillV2: Precise and Interactive Image Editing with Layered Visual Cues [106.02577891104079]
  We propose MagicQuill V2, a novel system that introduces a textbflayered composition paradigm to generative image editing.<n>Our method deconstructs creative intent into a stack of controllable visual cues.
  arXiv  Detail & Related papers  (2025-12-02T18:59:58Z)
• A Two-Stage System for Layout-Controlled Image Generation using Large Language Models and Diffusion Models [0.0]
  Text-to-image diffusion models exhibit remarkable generative capabilities, but lack precise control over object counts and spatial arrangements.<n>This work introduces a two-stage system to address these compositional limitations.<n>The first stage employs a Large Language Model (LLM) to generate a structured layout from a list of objects.<n>The second stage uses a layout-conditioned diffusion model to synthesize a photorealistic image adhering to this layout.
  arXiv  Detail & Related papers  (2025-11-10T09:40:48Z)
• DisCo-Layout: Disentangling and Coordinating Semantic and Physical Refinement in a Multi-Agent Framework for 3D Indoor Layout Synthesis [76.7196710324494]
  3D indoor layout synthesis is crucial for creating virtual environments.<n>DisCo is a novel framework that disentangles and coordinates physical and semantic refinement.
  arXiv  Detail & Related papers  (2025-10-02T16:30:37Z)
• CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation [78.21134311493303]
  Diffusion models have been recognized for their ability to generate images that are not only visually appealing but also of high artistic quality.<n> layout-to-image generation has been proposed to leverage region-specific positions and descriptions to enable more precise and controllable generation.<n>We present a systematic solution that integrates the layout model, dataset, and planner for creative layout-to-image generation.
  arXiv  Detail & Related papers  (2024-12-05T04:09:47Z)
• SceneComposer: Any-Level Semantic Image Synthesis [80.55876413285587]
  We propose a new framework for conditional image synthesis from semantic layouts of any precision levels. The framework naturally reduces to text-to-image (T2I) at the lowest level with no shape information, and it becomes segmentation-to-image (S2I) at the highest level. We introduce several novel techniques to address the challenges coming with this new setup.
  arXiv  Detail & Related papers  (2022-11-21T18:59:05Z)
• Geometry Aligned Variational Transformer for Image-conditioned Layout Generation [38.747175229902396]
  We propose an Image-Conditioned Variational Transformer (ICVT) that autoregressively generates various layouts in an image. First, self-attention mechanism is adopted to model the contextual relationship within layout elements, while cross-attention mechanism is used to fuse the visual information of conditional images. We construct a large-scale advertisement poster layout designing dataset with delicate layout and saliency map annotations.
  arXiv  Detail & Related papers  (2022-09-02T07:19:12Z)
• Interactive Image Synthesis with Panoptic Layout Generation [14.1026819862002]
  We propose Panoptic Layout Generative Adversarial Networks (PLGAN) to address this challenge. PLGAN employs panoptic theory which distinguishes object categories between "stuff" with amorphous boundaries and "things" with well-defined shapes. We experimentally compare our PLGAN with state-of-the-art layout-based models on the COCO-Stuff, Visual Genome, and Landscape datasets.
  arXiv  Detail & Related papers  (2022-03-04T02:45:27Z)
• Semantic Palette: Guiding Scene Generation with Class Proportions [34.746963256847145]
  We introduce a conditional framework with novel architecture designs and learning objectives, which effectively accommodates class proportions to guide the scene generation process. Thanks to the semantic control, we can produce layouts close to the real distribution, helping enhance the whole scene generation process. We demonstrate the merit of our approach for data augmentation: semantic segmenters trained on real layout-image pairs outperform models only trained on real pairs.
  arXiv  Detail & Related papers  (2021-06-03T07:04:00Z)
• TransVG: End-to-End Visual Grounding with Transformers [102.11922622103613]
  We present a transformer-based framework for visual grounding, namely TransVG, to address the task of grounding a language query to an image. We show that the complex fusion modules can be replaced by a simple stack of transformer encoder layers with higher performance.
  arXiv  Detail & Related papers  (2021-04-17T13:35:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.