MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon tasks

• URL: http://arxiv.org/abs/2603.11554v1
• Date: Thu, 12 Mar 2026 05:22:42 GMT
• Title: MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon tasks
• Abstract summary: We introduce MANSION, the first language-driven framework for generating building-scale, multi-floor 3D environments.<n>Mansion generates realistic, navigable whole-building structures with diverse, human-friendly scenes.<n>We release MansionWorld, a dataset of over 1,000 diverse buildings ranging from hospitals to offices.
• Score: 20.641188237918936
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world robotic tasks are long-horizon and often span multiple floors, demanding rich spatial reasoning. However, existing embodied benchmarks are largely confined to single-floor in-house environments, failing to reflect the complexity of real-world tasks. We introduce MANSION, the first language-driven framework for generating building-scale, multi-floor 3D environments. Being aware of vertical structural constraints, MANSION generates realistic, navigable whole-building structures with diverse, human-friendly scenes, enabling the development and evaluation of cross-floor long-horizon tasks. Building on this framework, we release MansionWorld, a dataset of over 1,000 diverse buildings ranging from hospitals to offices, alongside a Task-Semantic Scene Editing Agent that customizes these environments using open-vocabulary commands to meet specific user needs. Benchmarking reveals that state-of-the-art agents degrade sharply in our settings, establishing MANSION as a critical testbed for the next generation of spatial reasoning and planning.

Related papers

• SceneConductor: 3D Scene Generation from a Single Image with Multi-Agent Orchestration [32.39337008619354]
  We propose a multi-agent orchestration framework that decomposes single-image 3D scene generation into three structured stages.<n>We introduce a geometry-aware layout predictor supervised by sparse geometric priors derived from point maps.<n>Our method consistently outperforms prior approaches in geometric accuracy, spatial consistency, and perceptual realism.
  arXiv  Detail & Related papers  (2026-06-07T01:38:39Z)
• SDesc3D: Towards Layout-Aware 3D Indoor Scene Generation from Short Descriptions [71.54559024212976]
  3D indoor scene generation conditioned on short textual descriptions provides a promising avenue for interactive 3D environment construction.<n>Existing works suffer from poor physical plausibility and insufficient detail richness in such semantic condensation cases.<n>We propose SDesc3D, a short-text conditioned 3D indoor scene generation framework, that leverages multi-view structural priors and regional functionality implications.
  arXiv  Detail & Related papers  (2026-04-02T12:33:01Z)
• MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation [56.30931340537373]
  MolmoSpaces is a fully open ecosystem to support benchmarking of robot policies.<n>MolmoSpaces consists of over 230k diverse indoor environments.<n>MolmoSpaces-Bench is a benchmark suite of 8 tasks in which robots interact with our diverse scenes and richly annotated objects.
  arXiv  Detail & Related papers  (2026-02-11T20:16:31Z)
• Queryable 3D Scene Representation: A Multi-Modal Framework for Semantic Reasoning and Robotic Task Planning [28.803789915686398]
  3D Queryable Scene Representation (3D QSR) is a framework built on multimedia data that unifies three complementary 3D representations.<n>Built on an object-centric design, the framework integrates with large vision-language models to enable semantic queryability.<n>Results demonstrate the framework's ability to facilitate scene understanding and integrate spatial and semantic reasoning.
  arXiv  Detail & Related papers  (2025-09-24T12:53:32Z)
• HLG: Comprehensive 3D Room Construction via Hierarchical Layout Generation [31.010614667725843]
  Hierarchical Layout Generation (HLG) is a novel method for fine-grained 3D scene generation.<n>HLG is the first to adopt a coarse-to-fine hierarchical approach, refining scene layouts from large-scale furniture placement to intricate object arrangements.<n>We show superior performance in generating realistic indoor scenes compared to existing methods.
  arXiv  Detail & Related papers  (2025-08-25T09:32:57Z)
• SeqAffordSplat: Scene-level Sequential Affordance Reasoning on 3D Gaussian Splatting [85.87902260102652]
  We introduce the novel task of Sequential 3D Gaussian Affordance Reasoning.<n>We then propose SeqSplatNet, an end-to-end framework that directly maps an instruction to a sequence of 3D affordance masks.<n>Our method sets a new state-of-the-art on our challenging benchmark, effectively advancing affordance reasoning from single-step interactions to complex, sequential tasks at the scene level.
  arXiv  Detail & Related papers  (2025-07-31T17:56:55Z)
• Agentic 3D Scene Generation with Spatially Contextualized VLMs [67.31920821192323]
  We introduce a new paradigm that enables vision-language models to generate, understand, and edit complex 3D environments.<n>We develop an agentic 3D scene generation pipeline in which the VLM iteratively reads from and updates the spatial context.<n>Results show that our framework can handle diverse and challenging inputs, achieving a level of generalization not observed in prior work.
  arXiv  Detail & Related papers  (2025-05-26T15:28:17Z)
• WorldCraft: Photo-Realistic 3D World Creation and Customization via LLM Agents [67.31920821192323]
  We introduce WorldCraft, a system where large language model (LLM) agents leverage procedural generation to create scenes populated with objects.<n>In our framework, a coordinator agent manages the overall process and works with two specialized LLM agents to complete the scene creation.<n>Our pipeline incorporates a trajectory control agent, allowing users to animate the scene and operate the camera through natural language interactions.
  arXiv  Detail & Related papers  (2025-02-21T17:18:30Z)
• Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting [47.014044892025346]
  Architect is a generative framework that creates complex and realistic 3D embodied environments leveraging diffusion-based 2D image inpainting. Our pipeline is further extended to a hierarchical and iterative inpainting process to continuously generate placement of large furniture and small objects to enrich the scene.
  arXiv  Detail & Related papers  (2024-11-14T22:15:48Z)
• Embodied Task Planning with Large Language Models [86.63533340293361]
  We propose a TAsk Planing Agent (TaPA) in embodied tasks for grounded planning with physical scene constraint. During inference, we discover the objects in the scene by extending open-vocabulary object detectors to multi-view RGB images collected in different achievable locations. Experimental results show that the generated plan from our TaPA framework can achieve higher success rate than LLaVA and GPT-3.5 by a sizable margin.
  arXiv  Detail & Related papers  (2023-07-04T17:58:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.