Map2World: Segment Map Conditioned Text to 3D World Generation

• URL: http://arxiv.org/abs/2605.00781v1
• Date: Fri, 01 May 2026 16:56:49 GMT
• Title: Map2World: Segment Map Conditioned Text to 3D World Generation
• Abstract summary: 3D world generation is essential for immersive content creation or autonomous driving simulation.<n>Recent advances in 3D world generation have shown promising results; however, these methods are constrained by grid layouts.<n>We introduce a novel framework, Map2World, that first enables 3D world generation conditioned on user-defined segment maps of arbitrary shapes and scales.
• Score: 64.2098235580955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: 3D world generation is essential for applications such as immersive content creation or autonomous driving simulation. Recent advances in 3D world generation have shown promising results; however, these methods are constrained by grid layouts and suffer from inconsistencies in object scale throughout the entire world. In this work, we introduce a novel framework, Map2World, that first enables 3D world generation conditioned on user-defined segment maps of arbitrary shapes and scales, ensuring global-scale consistency and flexibility across expansive environments. To further enhance the quality, we propose a detail enhancer network that generates fine details of the world. The detail enhancer enables the addition of fine-grained details without compromising overall scene coherence by incorporating global structure information. We design the entire pipeline to leverage strong priors from asset generators, achieving robust generalization across diverse domains, even under limited training data for scene generation. Extensive experiments demonstrate that our method significantly outperforms existing approaches in user-controllability, scale consistency, and content coherence, enabling users to generate 3D worlds under more complex conditions.

Related papers

• WorldClaw: Agentic 3D Open-World Generation at Scale [20.019718140328347]
  We present WorldClaw, a fully agentic, coarse-to-fine framework for open-world 3D scene generation.<n>Planning agents translate a text prompt into a structured specification of regions, terrain, assets, materials, and spatial relations.<n>WorldClaw builds a globally coherent terrain foundation from semantic layouts, reusable assets, generative or procedural materials, and a region-aware height field.
  arXiv  Detail & Related papers  (2026-08-05T15:46:38Z)
• Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving [51.90209659403234]
  This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving.<n>For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries.<n>For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning.
  arXiv  Detail & Related papers  (2026-05-18T09:46:16Z)
• WorldFlow3D: Flowing Through 3D Distributions for Unbounded World Generation [33.369156362316055]
  Unbounded 3D world generation is emerging as a foundational task for scene modeling in computer vision, graphics, and robotics.<n>We present WorldFlow3D, a novel method capable of generating unbounded 3D worlds.
  arXiv  Detail & Related papers  (2026-03-31T00:08:17Z)
• Scene Generation at Absolute Scale: Utilizing Semantic and Geometric Guidance From Text for Accurate and Interpretable 3D Indoor Scene Generation [5.189354278537469]
  GuidedSceneGen produces metrically accurate, globally consistent, and semantically interpretable indoor scenes.<n>Our approach maintains an absolute world coordinate frame throughout the entire generation process.
  arXiv  Detail & Related papers  (2026-03-14T11:46:27Z)
• WorldGrow: Generating Infinite 3D World [75.81531067447203]
  We tackle the challenge of generating the infinitely extendable 3D world -- large, continuous environments with coherent geometry and realistic appearance.<n>We propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis.<n>Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity.
  arXiv  Detail & Related papers  (2025-10-24T17:39:52Z)
• Terra: Explorable Native 3D World Model with Point Latents [74.90179419859415]
  We present Terra, a native 3D world model that represents and generates explorable environments in an intrinsic 3D latent space.<n>Specifically, we propose a novel point-to-Gaussian variational autoencoder (P2G-VAE) that encodes 3D inputs into a latent point representation.<n>We then introduce a sparse point flow matching network (SPFlow) for generating the latent point representation, which simultaneously denoises the positions and features of the point latents.
  arXiv  Detail & Related papers  (2025-10-16T17:59:56Z)
• Graph-Guided Dual-Level Augmentation for 3D Scene Segmentation [21.553363236403822]
  3D point cloud segmentation aims to assign semantic labels to individual points in a scene for fine-grained spatial understanding.<n>Existing methods typically adopt data augmentation to alleviate the burden of large-scale annotation.<n>We propose a graph-guided data augmentation framework with dual-level constraints for realistic 3D scene synthesis.
  arXiv  Detail & Related papers  (2025-07-30T13:25:36Z)
• SynCity: Training-Free Generation of 3D Worlds [107.69875149880679]
  We propose SynCity, a training- and optimization-free approach to generating 3D worlds from textual descriptions.<n>We show how 3D and 2D generators can be combined to generate ever-expanding scenes.
  arXiv  Detail & Related papers  (2025-03-20T17:59:40Z)
• CULTURE3D: A Large-Scale and Diverse Dataset of Cultural Landmarks and Terrains for Gaussian-Based Scene Rendering [16.614203937535205]
  Current state-of-the-art 3D reconstruction models face limitations in building extra-large scale outdoor scenes.<n>In this paper, we present a extra-large fine-grained dataset with 10 billion points composed of 41,006 drone-captured high-resolution aerial images.<n>Compared to existing datasets, ours offers significantly larger scale and higher detail, uniquely suited for fine-grained 3D applications.
  arXiv  Detail & Related papers  (2025-01-12T20:36:39Z)
• World-Consistent Data Generation for Vision-and-Language Navigation [33.13590164890286]
  Vision-and-Language Navigation (VLN) is a challenging task that requires an agent to navigate through photorealistic environments following natural-language instructions.<n>One main obstacle existing in VLN is data scarcity, leading to poor generalization performance over unseen environments.<n>We propose the world-consistent data generation (WCGEN), an efficacious data-augmentation framework satisfying both diversity and world-consistency.
  arXiv  Detail & Related papers  (2024-12-09T11:40:54Z)
• Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation [64.07560335451723]
  CoSER is a novel consistent dense Multiview Text-to-Image Generator for Text-to-3D. It achieves both efficiency and quality by meticulously learning neighbor-view coherence. It aggregates information along motion paths explicitly defined by physical principles to refine details.
  arXiv  Detail & Related papers  (2024-08-23T15:16:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.