WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes

• URL: http://arxiv.org/abs/2605.15843v1
• Date: Fri, 15 May 2026 10:56:23 GMT
• Title: WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes
• Abstract summary: We present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes.<n>WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting.<n> Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.
• Score: 15.223046632151364
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent 3D world modeling systems based on generative scene synthesis, such as Marble, can create coherent and explorable 3D environments, yet their outputs are typically static monolithic assets with limited editability and physical interaction. This restricts their use in immersive content creation and embodied simulation, where generated worlds must be actively modified and manipulated. To tackle this challenge, we present WorldAct, a framework that converts static generated 3D worlds into editable and interaction-ready scenes. WorldAct uses a multimodal agent to guide scene decomposition, identify actionable objects, reconstruct geometrically aligned object-level meshes for interaction, and restore the residual background via 3D inpainting. The resulting scenes support object-level editing, collision-aware manipulation, and embodied task execution while preserving global scene coherence. Experiments show that WorldAct enables richer interaction scenarios than the original generated scenes, suggesting a practical path toward editable and interactive 3D world models.

Related papers

• WorldSculpt: Generating Compositional Worlds from Grounded Videos [50.427204240556854]
  We study the problem of generating a compositional 3D representation of a cluttered scene containing hundreds of objects.<n>This task is challenging in densely cluttered scenes, where objects heavily occlude one another and each view reveals only a fraction of their geometry.<n>We show that complex scenes with hundreds of objects can instead be generated compositionally by adapting a strong single-object 3D generative prior to multi-view observations.
  arXiv  Detail & Related papers  (2026-09-04T17:59:35Z)
• SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation [50.175987640001644]
  We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text.<n>We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes.
  arXiv  Detail & Related papers  (2026-07-02T06:29:18Z)
• SceneCode: Executable World Programs for Editable Indoor Scenes with Articulated Objects [69.20984454755512]
  Indoor scene synthesis underpins embodied AI, robotic manipulation, and simulation-based policy evaluation.<n>Existing pipelines represent generated content as static meshes and inherit articulation only from curated asset libraries.<n>We present SceneCode, a framework that compiles a natural language prompt into an executable, code-driven indoor world.
  arXiv  Detail & Related papers  (2026-05-19T09:31:04Z)
• WorldGen: From Text to Traversable and Interactive 3D Worlds [87.95088818329403]
  We introduce WorldGen, a system that enables the automatic creation of large-scale, interactive 3D worlds directly from text prompts.<n>Our approach transforms natural language descriptions into fully textured environments that can be immediately explored or edited within standard game engines.<n>This work represents a step towards accessible, generative world-building at scale, advancing the frontier of 3D generative AI for applications in gaming, simulation, and immersive social environments.
  arXiv  Detail & Related papers  (2025-11-20T22:13:18Z)
• ArtiWorld: LLM-Driven Articulation of 3D Objects in Scenes [43.19849355456126]
  ArtiWorld is a scene-aware pipeline that localizes candidate articulable objects from textual scene descriptions.<n>At the core of this pipeline is Arti4URDF, which leverages 3D point cloud and prior knowledge of a large language model.<n>We evaluate ArtiWorld at three levels: 3D simulated objects, full 3D simulated scenes, and real-world scan scenes.
  arXiv  Detail & Related papers  (2025-11-17T04:59:21Z)
• MorphoSim: An Interactive, Controllable, and Editable Language-guided 4D World Simulator [42.06769509897601]
  We introduce MorphoSim, a language guided framework that generates 4D scenes with multi-view consistency and object-level controls.<n>The framework integrates trajectory-guided generation with feature field dis tillation, allowing edits to be applied interactively without full re-generation.
  arXiv  Detail & Related papers  (2025-10-05T22:55:17Z)
• NeoWorld: Neural Simulation of Explorable Virtual Worlds via Progressive 3D Unfolding [46.79724166827757]
  We introduce NeoWorld, a framework for generating interactive 3D virtual worlds from a single input image.<n>Inspired by the on-demand worldbuilding concept in the science fiction novel Simulacron-3 (1964), our system constructs expansive environments.
  arXiv  Detail & Related papers  (2025-09-29T08:24:28Z)
• HiScene: Creating Hierarchical 3D Scenes with Isometric View Generation [50.206100327643284]
  HiScene is a novel hierarchical framework that bridges the gap between 2D image generation and 3D object generation.<n>We generate 3D content that aligns with 2D representations while maintaining compositional structure.
  arXiv  Detail & Related papers  (2025-04-17T16:33:39Z)
• Dynamic Scene Understanding through Object-Centric Voxelization and Neural Rendering [57.895846642868904]
  We present a 3D generative model named DynaVol-S for dynamic scenes that enables object-centric learning.<n>voxelization infers per-object occupancy probabilities at individual spatial locations.<n>Our approach integrates 2D semantic features to create 3D semantic grids, representing the scene through multiple disentangled voxel grids.
  arXiv  Detail & Related papers  (2024-07-30T15:33:58Z)
• Zero-Shot Multi-Object Scene Completion [59.325611678171974]
  We present a 3D scene completion method that recovers the complete geometry of multiple unseen objects in complex scenes from a single RGB-D image. Our method outperforms the current state-of-the-art on both synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2024-03-21T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.