LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restriction

• URL: http://arxiv.org/abs/2208.08037v2
• Date: Fri, 24 Mar 2023 08:31:19 GMT
• Title: LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restriction
• Authors: Zhaoyun Jiang, Jiaqi Guo, Shizhao Sun, Huayu Deng, Zhongkai Wu, Vuksan Mijovic, Zijiang James Yang, Jian-Guang Lou, Dongmei Zhang
• Abstract summary: Conditional graphic layout generation is a challenging task that has not been well-studied yet. We propose a constraint serialization scheme, a sequence-to-sequence transformation, and a decoding space restriction strategy. Experiments demonstrate that LayoutFormer++ outperforms existing approaches on all the tasks in terms of both better generation quality and less constraint violation.
• Score: 37.6871815321083
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conditional graphic layout generation, which generates realistic layouts according to user constraints, is a challenging task that has not been well-studied yet. First, there is limited discussion about how to handle diverse user constraints flexibly and uniformly. Second, to make the layouts conform to user constraints, existing work often sacrifices generation quality significantly. In this work, we propose LayoutFormer++ to tackle the above problems. First, to flexibly handle diverse constraints, we propose a constraint serialization scheme, which represents different user constraints as sequences of tokens with a predefined format. Then, we formulate conditional layout generation as a sequence-to-sequence transformation, and leverage encoder-decoder framework with Transformer as the basic architecture. Furthermore, to make the layout better meet user requirements without harming quality, we propose a decoding space restriction strategy. Specifically, we prune the predicted distribution by ignoring the options that definitely violate user constraints and likely result in low-quality layouts, and make the model samples from the restricted distribution. Experiments demonstrate that LayoutFormer++ outperforms existing approaches on all the tasks in terms of both better generation quality and less constraint violation.

Related papers

• CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusion [21.958752304572553]
  Existing models face two main challenges that limits their adoption in practice. Most existing models focus on generating labels and coordinates, while real layouts contain a range of style properties. We propose a novel framework, CoLay, that integrates multiple condition types and generates complex layouts with diverse style properties.
  arXiv  Detail & Related papers  (2024-05-18T17:30:48Z)
• Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints [53.66698106829144]
  We propose a unified model to handle a broad range of layout generation tasks. The model is based on continuous diffusion models. Experiment results show that LACE produces high-quality layouts.
  arXiv  Detail & Related papers  (2024-02-07T11:12:41Z)
• A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptions [50.469491454128246]
  We use text as the guidance to create graphic layouts, i.e., Text-to-labeled, aiming to lower the design barriers. Text-to-labeled is a challenging task, because it needs to consider the implicit, combined, and incomplete constraints from text. We present a two-stage approach, named parse-then-place, to address this problem.
  arXiv  Detail & Related papers  (2023-08-24T10:37:00Z)
• LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models [50.73105631853759]
  We present a novel generative model named LayoutDiffusion for automatic layout generation. It learns to reverse a mild forward process, in which layouts become increasingly chaotic with the growth of forward steps. It enables two conditional layout generation tasks in a plug-and-play manner without re-training and achieves better performance than existing methods.
  arXiv  Detail & Related papers  (2023-03-21T04:41:02Z)
• COFS: Controllable Furniture layout Synthesis [40.68096097121981]
  Many existing methods tackle this problem as a sequence generation problem which imposes a specific ordering on the elements of the layout. We propose COFS, an architecture based on standard transformer architecture blocks from language modeling. Our model consistently outperforms other methods which we verify by performing quantitative evaluations.
  arXiv  Detail & Related papers  (2022-05-29T13:31:18Z)
• Constrained Graphic Layout Generation via Latent Optimization [17.05026043385661]
  We generate graphic layouts that can flexibly incorporate design semantics, either specified implicitly or explicitly by a user. Our approach builds on a generative layout model based on a Transformer architecture, and formulates the layout generation as a constrained optimization problem. We show in the experiments that our approach is capable of generating realistic layouts in both constrained and unconstrained generation tasks with a single model.
  arXiv  Detail & Related papers  (2021-08-02T13:04:11Z)
• Generative Layout Modeling using Constraint Graphs [37.78500605563527]
  We propose a new generative model for layout generation. First, we generate the layout elements as nodes in a layout graph. Second, we compute constraints between layout elements as edges in the layout graph. Third, we solve for the final layout using constrained optimization.
  arXiv  Detail & Related papers  (2020-11-26T18:18:37Z)
• An Integer Linear Programming Framework for Mining Constraints from Data [81.60135973848125]
  We present a general framework for mining constraints from data. In particular, we consider the inference in structured output prediction as an integer linear programming (ILP) problem. We show that our approach can learn to solve 9x9 Sudoku puzzles and minimal spanning tree problems from examples without providing the underlying rules.
  arXiv  Detail & Related papers  (2020-06-18T20:09:53Z)
• ORCSolver: An Efficient Solver for Adaptive GUI Layout with OR-Constraints [63.59902335363947]
  ORCr is a novel solving technique for adaptive ORC layouts based on a branch-and-bound approach with preprocessing. We demonstrate that ORCr simplifies ORC specifications at runtime and our approach can solve ORC layout specifications efficiently at near-interactive rates.
  arXiv  Detail & Related papers  (2020-02-23T15:46:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.