HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models

• URL: http://arxiv.org/abs/2604.10772v1
• Date: Sun, 12 Apr 2026 18:45:40 GMT
• Title: HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models
• Abstract summary: 3D layout generation and editing play a crucial role in Embodied AI and immersive VR interaction.<n>We present HOG that enables text-driven hierarchical scene generation, optimization and real-time scene editing.
• Score: 12.672507892169477
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: 3D layout generation and editing play a crucial role in Embodied AI and immersive VR interaction. However, manual creation requires tedious labor, while data-driven generation often lacks diversity. The emergence of large models introduces new possibilities for 3D scene synthesis. We present HOG-Layout that enables text-driven hierarchical scene generation, optimization and real-time scene editing with large language models (LLMs) and vision-language models (VLMs). HOG-Layout improves scene semantic consistency and plausibility through retrieval-augmented generation (RAG) technology, incorporates an optimization module to enhance physical consistency, and adopts a hierarchical representation to enhance inference and optimization, achieving real-time editing. Experimental results demonstrate that HOG-Layout produces more reasonable environments compared with existing baselines, while supporting fast and intuitive scene editing.

Related papers

• Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models [57.59888019135289]
  Hash-Atlas network reformulates 3D scene editing as operations on 2D atlas images.<n>We introduce a dialogue-based 3D scene editing approach, termed CE3D++.<n>Results demonstrate that CE3D++ effectively integrates multiple visual models to achieve diverse visual editing effects.
  arXiv  Detail & Related papers  (2026-08-29T08:25:58Z)
• SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions [71.54559024212976]
  3D indoor scene generation conditioned on short textual descriptions provides a promising avenue for interactive 3D environment construction.<n>Existing works suffer from poor physical plausibility and insufficient detail richness in such semantic condensation cases.<n>We propose SDesc3D, a short-text conditioned 3D indoor scene generation framework, that leverages multi-view structural priors and regional functionality implications.
  arXiv  Detail & Related papers  (2026-04-02T12:33:01Z)
• LAYOUTDREAMER: Physics-guided Layout for Text-to-3D Compositional Scene Generation [5.424048651554831]
  We introduce a framework that leverages 3D Gaussian Splatting (3DGS) to facilitate high-quality, physically consistent compositional scene generation guided by text.<n>Specifically, given a text prompt, we convert it into a directed scene graph and adaptively adjust the density and layout of the initial compositional 3D Gaussians.<n>By extracting directed dependencies from the scene graph, we tailor physical and layout energy to ensure both realism and flexibility.
  arXiv  Detail & Related papers  (2025-02-04T02:51:37Z)
• EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting [3.9006270555948133]
  We propose EditSplat, a text-driven 3D scene editing framework that integrates Multi-view Fusion Guidance (MFG) and Attention-Guided Trimming (AGT)<n>Our MFG ensures multi-view consistency by incorporating essential multi-view information into the diffusion process.<n>Our AGT utilizes the explicit representation of 3DGS to selectively prune and optimize 3D Gaussians, enhancing optimization efficiency and enabling precise, semantically rich local editing.
  arXiv  Detail & Related papers  (2024-12-16T07:56:04Z)
• LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models [57.92316645992816]
  Spatial reasoning is a fundamental aspect of human cognition, enabling intuitive understanding and manipulation of objects in three-dimensional space.<n>We introduce LayoutVLM, a framework and scene layout representation that exploits the semantic knowledge of Vision-Language Models (VLMs)<n>We demonstrate that fine-tuning VLMs with the proposed scene layout representation extracted from existing scene datasets can improve their reasoning performance.
  arXiv  Detail & Related papers  (2024-12-03T06:15:04Z)
• EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing [114.14164860467227]
  We propose EditRoom, a framework capable of executing a variety of layout edits through natural language commands.<n>Specifically, EditRoom leverages Large Language Models (LLMs) for command planning and generates target scenes.<n>We have developed an automatic pipeline to augment existing 3D scene datasets and introduced EditRoom-DB, a large-scale dataset with 83k editing pairs.
  arXiv  Detail & Related papers  (2024-10-03T17:42:24Z)
• GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting [52.150502668874495]
  We present GALA3D, generative 3D GAussians with LAyout-guided control, for effective compositional text-to-3D generation. GALA3D is a user-friendly, end-to-end framework for state-of-the-art scene-level 3D content generation and controllable editing.
  arXiv  Detail & Related papers  (2024-02-11T13:40:08Z)
• Towards Language-guided Interactive 3D Generation: LLMs as Layout Interpreter with Generative Feedback [20.151147653552155]
  Large Language Models (LLMs) have demonstrated impressive reasoning, conversational, and zero-shot generation abilities. We propose a novel language-guided interactive 3D generation system, dubbed LI3D, that integrates LLMs as a 3D layout interpreter. Our system also incorporates LLaVA, a large language and vision assistant, to provide generative feedback from the visual aspect for improving the visual quality of generated content.
  arXiv  Detail & Related papers  (2023-05-25T07:43:39Z)
• LayoutGPT: Compositional Visual Planning and Generation with Large Language Models [98.81962282674151]
  Large Language Models (LLMs) can serve as visual planners by generating layouts from text conditions. We propose LayoutGPT, a method to compose in-context visual demonstrations in style sheet language.
  arXiv  Detail & Related papers  (2023-05-24T17:56:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.