SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation

• URL: http://arxiv.org/abs/2604.27555v1
• Date: Thu, 30 Apr 2026 08:04:42 GMT
• Title: SpatialGrammar: A Domain-Specific Language for LLM-Based 3D Indoor Scene Generation
• Abstract summary: We propose SpatialGrammar, a domain-specific language that represents gravity-aligned indoor layouts as BEV grid placements with deterministic compilation to valid 3D geometry.<n>We also develop SG-Agent, a closed-loop system that uses compiler feedback to iteratively refine scenes and enforce collision constraints, and SG-Mini, a 104M- parameter model trained entirely on compiler-validated synthetic data.
• Score: 19.357730085947523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatically generating interactive 3D indoor scenes from natural language is crucial for virtual reality, gaming, and embodied AI. However, existing LLM-based approaches often suffer from spatial errors and collisions, in part because common scene representations-raw coordinates or verbose code-are difficult for models to reason about 3D spatial relationships and physical constraints. We propose SpatialGrammar, a domain-specific language that represents gravity-aligned indoor layouts as BEV grid placements with deterministic compilation to valid 3D geometry, enabling verifiable constraint checking. Building on this representation, we develop (1) SG-Agent, a closed-loop system that uses compiler feedback to iteratively refine scenes and enforce collision constraints, and (2) SG-Mini, a 104M-parameter model trained entirely on compiler-validated synthetic data. Across 159 test scenes spanning five scenarios of different complexity, SG-Agent improves spatial fidelity and physical plausibility over prior methods, while SG-Mini performs competitively against larger LLM-based baselines on single-shot generation scenarios.

Related papers

• Global Graph-Validated Optimization for VLM-based 3D Indoor Scene Generation [15.117449876348774]
  We study open-vocabulary 3D indoor layout generation, synthesizes diverse physically plausible scenes from unlabeled 3D language instructions.<n>Our method achieves state-of-the-art performance in open-vocabulary 3D indoor layout generation.
  arXiv  Detail & Related papers  (2026-08-04T03:24:09Z)
• VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models [55.12929235609365]
  Visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision.<n>Existing implicit grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models.<n>We propose VEGA, a framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D.
  arXiv  Detail & Related papers  (2026-05-11T12:44:26Z)
• Scalable Object Relation Encoding for Better 3D Spatial Reasoning in Large Language Models [50.14156501544165]
  spatial reasoning focuses on locating target objects based on spatial relations in 3D scenes.<n>Previous approaches have attempted to inject 3D scene representations into the input space of Large Language Models.<n>We propose QuatRoPE, a novel positional embedding method with an input length that is linear to the number of objects.
  arXiv  Detail & Related papers  (2026-03-25T18:46:23Z)
• PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement [89.35154754765502]
  PhyScensis is an agent-based framework powered by a physics engine to produce physically plausible scene configurations.<n>Our framework preserves strong controllability over fine-grained textual descriptions and numerical parameters.<n> Experimental results show that our method outperforms prior approaches in scene complexity, visual quality, and physical accuracy.
  arXiv  Detail & Related papers  (2026-02-16T17:55:25Z)
• From Spatial to Actions: Grounding Vision-Language-Action Model in Spatial Foundation Priors [54.84863164684646]
  Existing vision-language-action (VLA) models act in 3D real-world but are typically built on 2D encoders.<n>In this work, we introduce FALCON, a novel paradigm that injects rich 3D spatial tokens into the action head.
  arXiv  Detail & Related papers  (2025-10-20T11:26:45Z)
• ReSem3D: Refinable 3D Spatial Constraints via Fine-Grained Semantic Grounding for Generalizable Robotic Manipulation [12.059517583878756]
  We propose ReSem3D, a unified manipulation framework for semantically diverse environments.<n>We show that ReSem3D performs diverse manipulation tasks under zero-shot conditions, exhibiting strong adaptability and generalization.
  arXiv  Detail & Related papers  (2025-07-24T10:07:31Z)
• RoomCraft: Controllable and Complete 3D Indoor Scene Generation [51.19602078504066]
  RoomCraft is a multi-stage pipeline that converts real images, sketches, or text descriptions into coherent 3D indoor scenes.<n>Our approach combines a scene generation pipeline with a constraint-driven optimization framework.<n>RoomCraft significantly outperforms existing methods in generating realistic, semantically coherent, and visually appealing room layouts.
  arXiv  Detail & Related papers  (2025-06-27T15:03:17Z)
• Agentic 3D Scene Generation with Spatially Contextualized VLMs [67.31920821192323]
  We introduce a new paradigm that enables vision-language models to generate, understand, and edit complex 3D environments.<n>We develop an agentic 3D scene generation pipeline in which the VLM iteratively reads from and updates the spatial context.<n>Results show that our framework can handle diverse and challenging inputs, achieving a level of generalization not observed in prior work.
  arXiv  Detail & Related papers  (2025-05-26T15:28:17Z)
• DeBaRA: Denoising-Based 3D Room Arrangement Generation [22.96293773013579]
  We introduce DeBaRA, a score-based model specifically tailored for precise, controllable and flexible arrangement generation in a bounded environment. We demonstrate that by focusing on spatial attributes of objects, a single trained DeBaRA model can be leveraged at test time to perform several downstream applications such as scene synthesis, completion and re-arrangement.
  arXiv  Detail & Related papers  (2024-09-26T23:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.