4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation

• URL: http://arxiv.org/abs/2608.26947v1
• Date: Thu, 27 Aug 2026 10:49:20 GMT
• Title: 4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation
• Abstract summary: Embodied agents need environments that are visually diverse, physically interactive, and changing over time.<n>We present 4D Synth, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment.<n>Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation.
• Score: 16.13796756164597
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large interactive scene collections, and recent 4D generators produce compelling visual dynamics. Combining these properties in one environment, however, still demands extensive manual effort, and the result is rarely editable or controllable enough to reuse at scale. We present 4DSynth, a controllable procedural system that turns a natural-language description, a blueprint mask, or a single photograph into an editable 4D environment with explicit geometry, animated actors, collision-free trajectories, and physics-ready simulation state. Multiple scene routes share one geometry-grounded representation, so the same pipeline handles animation, camera planning, rendering, and task generation. To validate the full pipeline, we construct 4DSynth-Nav, an interactive navigation benchmark generated entirely from 4DSynth's procedural scenes. Two vision-language models evaluated across three difficulty tiers both fail the majority of tasks and stall after early subtasks. The same procedural controllability that produces these environments also makes each failure reproducible and each difficulty axis independently tunable. This paper presents both a controllable generation pipeline and the scalable benchmark it enables, offering a practical foundation for developing and evaluating embodied agents.

Related papers

• SimWorlds: A Multi-Agent System for Dynamic 3D Scene Creation [50.175987640001644]
  We present SimWorlds: a multi-agent framework that produces dynamic, editable 4D scenes from text.<n>We also introduce 4DBuildBench, a benchmark for assessing both visual fidelity and physical consistency of the procedural dynamic 3D scenes.
  arXiv  Detail & Related papers  (2026-07-02T06:29:18Z)
• One Video, One World: Turning Monocular Video into Physical 4D Scenes [27.429882736058904]
  OVOW reconstructs emphinstance-level, simulation-ready 4D mesh scenes from a single monocular video.<n>We establish the first benchmark for emphstructured Video-to-4D evaluation, with metrics for geometric correctness, instance separation, and physical plausibility beyond visual fidelity.<n>OVOW attains the best overall layout and geometry accuracy and the lowest photometric and semantic error among all baselines.
  arXiv  Detail & Related papers  (2026-06-30T09:16:21Z)
• Syn4D: A Multiview Synthetic 4D Dataset [92.50132053229817]
  3D reconstruction and tracking of dynamic scenes from monocular video remains an important open challenge in computer vision.<n>To address this limitation, we introduce Syn4D, a multiview synthetic dynamic scenes.<n>Key feature of Syn4D is the ability to unproject any pixel into 3D.
  arXiv  Detail & Related papers  (2026-05-06T17:59:55Z)
• Asset Harvester: Extracting 3D Assets from Autonomous Driving Logs for Simulation [63.01022057888141]
  Asset Harvester is an image-to-3D model and end-to-end pipeline that converts sparse, in-thewild object observations from real driving logs into complete, simulation-ready assets.<n>SparseViewDiT is explicitly designed to address limited-angle views and other real-world data challenges.
  arXiv  Detail & Related papers  (2026-04-20T16:20:57Z)
• MorphoSim: An Interactive, Controllable, and Editable Language-guided 4D World Simulator [42.06769509897601]
  We introduce MorphoSim, a language guided framework that generates 4D scenes with multi-view consistency and object-level controls.<n>The framework integrates trajectory-guided generation with feature field dis tillation, allowing edits to be applied interactively without full re-generation.
  arXiv  Detail & Related papers  (2025-10-05T22:55:17Z)
• 4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives [115.67081491747943]
  Dynamic 3D scene representation and novel view synthesis are crucial for enabling AR/VR and metaverse applications.<n>We reformulate the reconstruction of a time-varying 3D scene as approximating its underlying 4D volume.<n>We derive several compact variants that effectively reduce the memory footprint to address its storage bottleneck.
  arXiv  Detail & Related papers  (2024-12-30T05:30:26Z)
• DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulation [54.02069690134526]
  We propose DrivingSphere, a realistic and closed-loop simulation framework. Its core idea is to build 4D world representation and generate real-life and controllable driving scenarios. By providing a dynamic and realistic simulation environment, DrivingSphere enables comprehensive testing and validation of autonomous driving algorithms.
  arXiv  Detail & Related papers  (2024-11-18T03:00:33Z)
• Trans4D: Realistic Geometry-Aware Transition for Compositional Text-to-4D Synthesis [60.853577108780414]
  Existing 4D generation methods can generate high-quality 4D objects or scenes based on user-friendly conditions. We propose Trans4D, a novel text-to-4D synthesis framework that enables realistic complex scene transitions. In experiments, Trans4D consistently outperforms existing state-of-the-art methods in generating 4D scenes with accurate and high-quality transitions.
  arXiv  Detail & Related papers  (2024-10-09T17:56:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.