LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models

• URL: http://arxiv.org/abs/2303.11589v2
• Date: Tue, 15 Aug 2023 06:55:06 GMT
• Title: LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models
• Authors: Junyi Zhang, Jiaqi Guo, Shizhao Sun, Jian-Guang Lou, Dongmei Zhang
• Abstract summary: We present a novel generative model named LayoutDiffusion for automatic layout generation. It learns to reverse a mild forward process, in which layouts become increasingly chaotic with the growth of forward steps. It enables two conditional layout generation tasks in a plug-and-play manner without re-training and achieves better performance than existing methods.
• Score: 50.73105631853759
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Creating graphic layouts is a fundamental step in graphic designs. In this work, we present a novel generative model named LayoutDiffusion for automatic layout generation. As layout is typically represented as a sequence of discrete tokens, LayoutDiffusion models layout generation as a discrete denoising diffusion process. It learns to reverse a mild forward process, in which layouts become increasingly chaotic with the growth of forward steps and layouts in the neighboring steps do not differ too much. Designing such a mild forward process is however very challenging as layout has both categorical attributes and ordinal attributes. To tackle the challenge, we summarize three critical factors for achieving a mild forward process for the layout, i.e., legality, coordinate proximity and type disruption. Based on the factors, we propose a block-wise transition matrix coupled with a piece-wise linear noise schedule. Experiments on RICO and PubLayNet datasets show that LayoutDiffusion outperforms state-of-the-art approaches significantly. Moreover, it enables two conditional layout generation tasks in a plug-and-play manner without re-training and achieves better performance than existing methods.

Related papers

• Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion [61.03681839276652]
  Diffusion Forcing is a new training paradigm where a diffusion model is trained to denoise a set of tokens with independent per-token noise levels. We apply Diffusion Forcing to sequence generative modeling by training a causal next-token prediction model to generate one or several future tokens.
  arXiv  Detail & Related papers  (2024-07-01T15:43:25Z)
• LayoutFlow: Flow Matching for Layout Generation [23.045325684880957]
  We propose an efficient flow-based model capable of generating high-quality layouts. Our method learns to gradually move, or flow, the elements of an initial sample until it reaches its final prediction.
  arXiv  Detail & Related papers  (2024-03-27T01:40:21Z)
• Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints [53.66698106829144]
  We propose a unified model to handle a broad range of layout generation tasks. The model is based on continuous diffusion models. Experiment results show that LACE produces high-quality layouts.
  arXiv  Detail & Related papers  (2024-02-07T11:12:41Z)
• Adversarial Supervision Makes Layout-to-Image Diffusion Models Thrive [21.49096276631859]
  Current L2I models either suffer from poor editability via text or weak alignment between the generated image and the input layout. We propose to integrate adversarial supervision into the conventional training pipeline of L2I diffusion models (ALDM) Specifically, we employ a segmentation-based discriminator which provides explicit feedback to the diffusion generator on the pixel-level alignment between the denoised image and the input layout.
  arXiv  Detail & Related papers  (2024-01-16T20:31:46Z)
• R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation [74.5598315066249]
  We probe into zero-shot grounded T2I generation with diffusion models. We propose a Region and Boundary (R&B) aware cross-attention guidance approach.
  arXiv  Detail & Related papers  (2023-10-13T05:48:42Z)
• A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptions [50.469491454128246]
  We use text as the guidance to create graphic layouts, i.e., Text-to-labeled, aiming to lower the design barriers. Text-to-labeled is a challenging task, because it needs to consider the implicit, combined, and incomplete constraints from text. We present a two-stage approach, named parse-then-place, to address this problem.
  arXiv  Detail & Related papers  (2023-08-24T10:37:00Z)
• PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layout [62.12447593298437]
  Content-aware visual-textual presentation layout aims at arranging spatial space on the given canvas for pre-defined elements. We propose design sequence formation (DSF) that reorganizes elements in layouts to imitate the design processes of human designers. A novel CNN-LSTM-based conditional generative adversarial network (GAN) is presented to generate proper layouts.
  arXiv  Detail & Related papers  (2023-03-28T12:48:36Z)
• LayoutDM: Discrete Diffusion Model for Controllable Layout Generation [27.955214767628107]
  Controllable layout generation aims at synthesizing plausible arrangement of element bounding boxes with optional constraints. In this work, we try to solve a broad range of layout generation tasks in a single model that is based on discrete state-space diffusion models. Our model, named LayoutDM, naturally handles the structured layout data in the discrete representation and learns to progressively infer a noiseless layout from the initial input.
  arXiv  Detail & Related papers  (2023-03-14T17:59:47Z)
• LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation [24.694298869398033]
  Our method trains efficiently, generates images with both high perceptual quality and layout alignment. Our method significantly outperforms other 10 generative models based on GANs, VQ-VAE, and diffusion models.
  arXiv  Detail & Related papers  (2023-02-16T14:20:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.