3D-Layout-R1: Structured Reasoning for Language-Instructed Spatial Editing

• URL: http://arxiv.org/abs/2603.22279v1
• Date: Mon, 23 Mar 2026 17:59:14 GMT
• Title: 3D-Layout-R1: Structured Reasoning for Language-Instructed Spatial Editing
• Abstract summary: We introduce a Structured Reasoning framework that performs text-conditioned spatial layout editing via scene-graph reasoning.<n>By explicitly guiding the reasoning process through structured relational representations, our approach improves both interpretability and control over spatial relationships.
• Score: 57.785328880266775
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large Language Models (LLMs) and Vision Language Models (VLMs) have shown impressive reasoning abilities, yet they struggle with spatial understanding and layout consistency when performing fine-grained visual editing. We introduce a Structured Reasoning framework that performs text-conditioned spatial layout editing via scene-graph reasoning. Given an input scene graph and a natural-language instruction, the model reasons over the graph to generate an updated scene graph that satisfies the text condition while maintaining spatial coherence. By explicitly guiding the reasoning process through structured relational representations, our approach improves both interpretability and control over spatial relationships. We evaluate our method on a new text-guided layout editing benchmark encompassing sorting, spatial alignment, and room-editing tasks. Our training paradigm yields an average 15% improvement in IoU and 25% reduction in center-distance error compared to Chain of Thought Fine-tuning (CoT-SFT) and vanilla GRPO baselines. Compared to SOTA zero-shot LLMs, our best models achieve up to 20% higher mIoU, demonstrating markedly improved spatial precision.

Related papers

• Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models [46.09809083890632]
  Scene Graph Thinking (SaGe) is a novel paradigm that enables fine-grained and structured visual reasoning through explicit scene-graph representations.<n>We construct 120K high-quality training data by sampling reasoning traces from scene graphs.<n>With curated data and graph-aligned training, our approach achieves significant improvements across eight multimodal benchmarks.
  arXiv  Detail & Related papers  (2026-07-07T01:00:51Z)
• SpaceEra++: A Unified Framework Towards 3D Spatial Reasoning in Video [88.79625979053873]
  We propose a novel framework, SpaceEra, for pre-trained vision-language models (VLMs)<n>We extend the original framework into a comprehensive system, termed SpaceEra++, which spans data construction, model design, training optimization, and prompting inference.<n>In addition, to enhance spatial reasoning, we develop SpaceAlign, which enforces pairwise object constraints by jointly exploiting absolute coordinates and relative spatial relations.
  arXiv  Detail & Related papers  (2026-07-02T06:56:29Z)
• SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching [2.4737219202679808]
  We propose SceneGrapher, a framework that reformulates 3D grounding as structured graph matching over a reconstructed 3D scene graph.<n>We demonstrate our framework through real-world deployment on a mobile robot, demonstrating robust spatial reasoning in long-horizon physical environments.
  arXiv  Detail & Related papers  (2026-05-20T22:30:51Z)
• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• Enhancing Spatial Reasoning in Vision-Language Models via Chain-of-Thought Prompting and Reinforcement Learning [0.42855555838080844]
  This study investigates the spatial reasoning capabilities of vision-language models (VLMs) through Chain-of-Thought prompting and reinforcement learning.<n>We find that simple CoT formats, where the model generates a reasoning step before the answer, can harm the model's original performance.<n>In contrast, structured multi-stage prompting based on scene graphs (SceneGraph CoT) significantly improves spatial reasoning accuracy.
  arXiv  Detail & Related papers  (2025-07-06T10:51:12Z)
• Visual Structures Helps Visual Reasoning: Addressing the Binding Problem in VLMs [9.406760867809124]
  This paper introduces Visual Input Structure for Enhanced Reasoning (VISER)<n>VISER is a simple, effective method that augments visual inputs with low-level spatial structures.<n>We empirically demonstrate substantial performance improvements across core visual reasoning tasks.
  arXiv  Detail & Related papers  (2025-06-27T11:44:40Z)
• Compile Scene Graphs with Reinforcement Learning [69.36723767339001]
  Next-token prediction is the fundamental principle for training large language models (LLMs)<n>We introduce R1-SGG, a multimodal LLM (M-LLM) initially trained via supervised fine-tuning (SFT) on the scene graph dataset.<n>We design a set of graph-centric rewards, including three recall-based variants -- Hard Recall, Hard Recall+Relax, and Soft Recall.
  arXiv  Detail & Related papers  (2025-04-18T10:46:22Z)
• LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Models [57.92316645992816]
  Spatial reasoning is a fundamental aspect of human cognition, enabling intuitive understanding and manipulation of objects in three-dimensional space.<n>We introduce LayoutVLM, a framework and scene layout representation that exploits the semantic knowledge of Vision-Language Models (VLMs)<n>We demonstrate that fine-tuning VLMs with the proposed scene layout representation extracted from existing scene datasets can improve their reasoning performance.
  arXiv  Detail & Related papers  (2024-12-03T06:15:04Z)
• Multi-View Empowered Structural Graph Wordification for Language Models [12.22063024099311]
  We introduce an end-to-end modality-aligning framework for LLM-graph alignment: Dual-Residual Vector Quantized-Variational AutoEncoder, namely Dr.E.<n>Our approach is purposefully designed to facilitate token-level alignment with LLMs, enabling an effective translation of the intrinsic'of graphs into comprehensible natural language.<n>Our framework ensures certain visual interpretability, efficiency, and robustness, marking the promising successful endeavor to achieve token-level alignment between LLMs and GNNs.
  arXiv  Detail & Related papers  (2024-06-19T16:43:56Z)
• Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models [3.86170450233149]
  We show that large vision-and-language models (VLMs) trained to match images with text lack fine-grained understanding of spatial relations. We propose an alternative fine-grained, compositional approach for recognizing and ranking spatial clauses.
  arXiv  Detail & Related papers  (2023-08-18T18:58:54Z)
• Language and Visual Entity Relationship Graph for Agent Navigation [54.059606864535304]
  Vision-and-Language Navigation (VLN) requires an agent to navigate in a real-world environment following natural language instructions. We propose a novel Language and Visual Entity Relationship Graph for modelling the inter-modal relationships between text and vision. Experiments show that by taking advantage of the relationships we are able to improve over state-of-the-art.
  arXiv  Detail & Related papers  (2020-10-19T08:25:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.