SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation

• URL: http://arxiv.org/abs/2607.01766v1
• Date: Thu, 02 Jul 2026 06:29:18 GMT
• Title: SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation
• Abstract summary: We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text.<n>We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes.
• Score: 50.175987640001644
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LLM agents are increasingly used to translate natural language into 3D scenes in a procedural way, but existing systems focus on static output. Dynamic 4D scenes from text alone, in which liquids flow, particles emit, rigid bodies cascade, and articulated mechanisms move, remain largely unexplored despite their value as editable content and as physics-grounded training data for video generation and embodied AI. Two challenges set the dynamic case apart from static text-to-scene work: an agent must jointly coordinate spatial layout, multiple physics solvers, temporal sequencing, camera, and lighting in a single coherent scene, and verifying motion correctness from rendered video is fundamentally harder than judging a single image. We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text, with Blender-specific procedural knowledge, a planner-coder-reviewer workflow driving a fixed ordered sequence of construction stages, a layered scene protocol enforced by a deterministic verifier, and a runtime-state inspection tool suite that catches mechanism failures the rendered image cannot reveal. We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes generated from text prompts. Experiments show that SimWorlds outperforms prior dynamic Blender generation baselines.

Related papers

• ESG: Generating Physically Consistent Dynamic 3D Scenes from Text Descriptions [19.29011683959245]
  We present a unified framework for generating physically consistent 3D scenes from text, with directly executable in Unreal Engine.<n>We use the emphEvolutive Scene Graph (ESG), which specifies entities with physical attributes, spatial relations, and event-driven timelines in a machine-checkable form.
  arXiv  Detail & Related papers  (2026-09-14T11:19:25Z)
• 4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation [16.13796756164597]
  Embodied agents need environments that are visually diverse, physically interactive, and changing over time.<n>We present 4D Synth, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment.<n>Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation.
  arXiv  Detail & Related papers  (2026-08-27T10:49:20Z)
• PhysLayer: Language-Guided Layered Animation with Depth-Aware Physics [17.206634450299536]
  Existing image-to-video generation methods often produce physically implausible motions and lack precise control over object dynamics.<n>We introduce PhysLayer, a novel framework enabling language-guided, depth-aware layered animation of static images.
  arXiv  Detail & Related papers  (2026-04-26T07:29:13Z)
• StoryBlender: Inter-Shot Consistent and Editable 3D Storyboard with Spatial-temporal Dynamics [56.8957922153694]
  We present StoryBlender, a grounded 3D storyboard generation framework governed by a Story-centric Reflection Scheme.<n>StoryBlender significantly improves consistency and editability over both diffusion-based and 3D-grounded baselines.
  arXiv  Detail & Related papers  (2026-04-01T07:24:31Z)
• PAT3D: Physics-Augmented Text-to-3D Scene Generation [47.18949891825537]
  PAT3D generates 3D objects, infers their spatial relations, and organizes them into a hierarchical scene tree.<n>A differentiable rigid-body simulator ensures realistic object interactions under gravity.<n>Experiments demonstrate that PAT3D substantially outperforms prior approaches in physical plausibility, semantic consistency, and visual quality.
  arXiv  Detail & Related papers  (2025-11-26T23:23:58Z)
• Drag4D: Align Your Motion with Text-Driven 3D Scene Generation [77.79131321983677]
  Drag4D is an interactive framework that integrates object motion control within text-driven 3D scene generation.<n>This framework enables users to define 3D trajectories for the 3D objects generated from a single image, seamlessly integrating them into a high-quality 3D background.
  arXiv  Detail & Related papers  (2025-09-26T05:23:45Z)
• 4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives [115.67081491747943]
  Dynamic 3D scene representation and novel view synthesis are crucial for enabling AR/VR and metaverse applications.<n>We reformulate the reconstruction of a time-varying 3D scene as approximating its underlying 4D volume.<n>We derive several compact variants that effectively reduce the memory footprint to address its storage bottleneck.
  arXiv  Detail & Related papers  (2024-12-30T05:30:26Z)
• Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives [70.32817882783608]
  We present an approach that produces a simple, compact, and actionable 3D world representation by means of 3D primitives. Unlike existing primitive decomposition methods that rely on 3D input data, our approach operates directly on images. We show that the resulting textured primitives faithfully reconstruct the input images and accurately model the visible 3D points.
  arXiv  Detail & Related papers  (2023-07-11T17:58:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.