SG-Layout: Structured Scene Graph-Guided Layout Generation with LLMs

• URL: http://arxiv.org/abs/2608.01106v1
• Date: Sun, 02 Aug 2026 09:11:38 GMT
• Title: SG-Layout: Structured Scene Graph-Guided Layout Generation with LLMs
• Abstract summary: We propose SG-, a graph-guided layout generation framework that explicitly incorporates structured spatial knowledge into large language models (LLMs)<n> Experimental results show that SG- improves geometric accuracy, consistency and reasoning over the compact open-source backbone.<n>Results highlight the effectiveness of graph-structured feature alignment for enhancing controllable layout generation.
• Score: 19.275150322459865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding and generating spatially coherent layouts from natural language remains a fundamental yet challenging task for large language models (LLMs). Existing LLMs often struggle to capture explicit geometric relationships and structural dependencies between objects. To address this issue, we propose SG-Layout, a graph-guided layout generation framework that explicitly incorporates structured spatial knowledge into LLMs. SG-Layout follows a two-stage training paradigm: (1) a graph-language feature alignment stage, where a relational graph encoder and a projector are trained to map scene-graph embeddings into the LLM's linguistic space; and (2) an instruction tuning stage, where LoRA-based adapters enable efficient fine-tuning for instruction-driven layout generation while keeping the backbone frozen. We evaluate SG-Layout on image layout generation, indoor scene synthesis and robotic object rearrangement tasks. Experimental results show that SG-Layout improves spatial reasoning accuracy and geometric consistency over the compact open-source backbone, with particularly clear advantages in relation-dense and compositionally complex scenes. These results highlight the effectiveness of graph-structured feature alignment for enhancing controllable layout generation.

Related papers

• GALA: Multimodal Graph Alignment for Bug Localization in Automated Program Repair [40.228155198574356]
  We propose GALA, a framework that shifts multimodal Automated Program Repair (APR) from implicit semantic guessing to explicit structural reasoning.<n>GALA operates in four stages: it first constructs an Image UI Graph to capture visual elements and their structural relationships; then performs file-level alignment by cross-referencing this UI graph with repository-level structures.<n>By enforcing both semantic and relational consistency across modalities, GALA establishes a highly accurate visual-to-code mapping.
  arXiv  Detail & Related papers  (2026-04-09T11:06:25Z)
• 3D-Layout-R1: Structured Reasoning for Language-Instructed Spatial Editing [57.785328880266775]
  We introduce a Structured Reasoning framework that performs text-conditioned spatial layout editing via scene-graph reasoning.<n>By explicitly guiding the reasoning process through structured relational representations, our approach improves both interpretability and control over spatial relationships.
  arXiv  Detail & Related papers  (2026-03-23T17:59:14Z)
• NAG: A Unified Native Architecture for Encoder-free Text-Graph Modeling in Language Models [33.49410203951687]
  We argue this approach is suboptimal for text-graphs.<n>NAG (Native Architecture for Graphs) is a unified framework that internalizes graph processing within the Language Models.<n>NAG achieves robust graph comprehension without the overhead of external encoders.
  arXiv  Detail & Related papers  (2026-01-30T07:22:11Z)
• DisCo-Layout: Disentangling and Coordinating Semantic and Physical Refinement in a Multi-Agent Framework for 3D Indoor Layout Synthesis [76.7196710324494]
  3D indoor layout synthesis is crucial for creating virtual environments.<n>DisCo is a novel framework that disentangles and coordinates physical and semantic refinement.
  arXiv  Detail & Related papers  (2025-10-02T16:30:37Z)
• LLMs as Layout Designers: Enhanced Spatial Reasoning for Content-Aware Layout Generation [27.377286708795506]
  LaySPA is a reinforcement learning-based framework that augments Large Language Models with explicit spatial reasoning capabilities for layout design.<n>We show that LaySPA substantially improves the generation of structurally valid and visually appealing layouts.
  arXiv  Detail & Related papers  (2025-09-21T03:02:59Z)
• LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models [57.92316645992816]
  Spatial reasoning is a fundamental aspect of human cognition, enabling intuitive understanding and manipulation of objects in three-dimensional space.<n>We introduce LayoutVLM, a framework and scene layout representation that exploits the semantic knowledge of Vision-Language Models (VLMs)<n>We demonstrate that fine-tuning VLMs with the proposed scene layout representation extracted from existing scene datasets can improve their reasoning performance.
  arXiv  Detail & Related papers  (2024-12-03T06:15:04Z)
• Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment [130.15775113897553]
  Finsta is a fine-grained structural-temporal alignment learning method. It consistently improves the existing 13 strong-tuning video-language models.
  arXiv  Detail & Related papers  (2024-06-27T15:23:36Z)
• A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptions [50.469491454128246]
  We use text as the guidance to create graphic layouts, i.e., Text-to-labeled, aiming to lower the design barriers. Text-to-labeled is a challenging task, because it needs to consider the implicit, combined, and incomplete constraints from text. We present a two-stage approach, named parse-then-place, to address this problem.
  arXiv  Detail & Related papers  (2023-08-24T10:37:00Z)
• LAW-Diffusion: Complex Scene Generation by Diffusion with Layouts [107.11267074981905]
  We propose a semantically controllable layout-AWare diffusion model, termed LAW-Diffusion. We show that LAW-Diffusion yields the state-of-the-art generative performance, especially with coherent object relations.
  arXiv  Detail & Related papers  (2023-08-13T08:06:18Z)
• LayoutGPT: Compositional Visual Planning and Generation with Large Language Models [98.81962282674151]
  Large Language Models (LLMs) can serve as visual planners by generating layouts from text conditions. We propose LayoutGPT, a method to compose in-context visual demonstrations in style sheet language.
  arXiv  Detail & Related papers  (2023-05-24T17:56:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.