WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory

• URL: http://arxiv.org/abs/2607.02517v1
• Date: Thu, 02 Jul 2026 17:59:59 GMT
• Title: WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory
• Abstract summary: WorldDirector is a highly controllable video world model framework.<n>Our framework explicitly decouples semantic motion orchestration from visual generation.<n>We successfully preserve the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.
• Score: 106.28234880163723
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation. By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experimental results demonstrate that our method supports the synthesis of complex and extended events with unprecedented controllability and persistent dynamic object memory. Project Page: https://worlddirector.github.io/

Related papers

• EchoWM: Open and Enterable Omnimodal World Models [71.49148067556887]
  EchoWM is an omnimodal world model for enterable generative media.<n>It responds to continuous navigation while jointly generating 720p video, environmental sound, music and speech.
  arXiv  Detail & Related papers  (2026-08-24T12:39:59Z)
• FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control [26.208912960696622]
  We propose a unified framework that standardizes video compositing as a trajectory-guided conditional generation task.<n>Our approach introduces three key designs: (1) a Unified Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to
  arXiv  Detail & Related papers  (2026-07-31T16:59:46Z)
• Directing the World: Fast Autoregressive Video Generation with Compositional Human-Camera Control [56.828569670698975]
  "Directing the World" is a fast autoregressive framework for controllable world-model video generation with compositional human-motion and camera-trajectory control.<n>Our key idea is to decouple control learning while preserving a unified autoregressive video prior.<n>For human motion control, we design a t-guided Dynamic Projection mechanism and a refined Motion-CFG strategy.
  arXiv  Detail & Related papers  (2026-06-26T11:08:09Z)
• Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds [22.4160057556312]
  Look-Before-Move is a camera planning framework that separates observation specification from motion execution.<n>We construct a dynamic 3D Story World Benchmark based on StoryBlender, covering 50 stories, 457 scenes, and 1585 shots with animated characters.<n>Experiments show that our framework improves subject perception, intent consistency, and trajectory quality over representative baselines.
  arXiv  Detail & Related papers  (2026-06-25T12:38:39Z)
• MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data [125.43597497646444]
  MetaWorld is a novel framework that scales multi-agent video world models to open-domain environments directly from single-view videos.<n>It achieves superior cross-view consistency and identity fidelity, establishing a highly scalable, physics-driven paradigm for multi-agent video world modeling.
  arXiv  Detail & Related papers  (2026-06-01T18:20:20Z)
• SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generation [11.470401259593944]
  SymphoMotion is a unified motion-control framework that governs camera trajectories and object dynamics within a single model.<n>We show that SymphoMotion significantly outperforms existing methods in visual fidelity, camera controllability, and object-motion accuracy.
  arXiv  Detail & Related papers  (2026-04-04T12:59:24Z)
• LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models [32.92934803081681]
  Recent generative video world models aim to simulate visual environment evolution, allowing an observer to interactively explore the scene via camera control.<n>They implicitly assume that the world only evolves within the observer's field of view.<n>Once an object leaves the observer's view, its state is "frozen" in memory, and revisiting the same region later often fails to reflect events that should have occurred in the meantime.<n>We propose LiveWorld, a novel framework that extends video world models to support persistent world evolution.
  arXiv  Detail & Related papers  (2026-03-07T10:31:39Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models [75.20168902300166]
  We propose TrackDiffusion, a novel video generation framework affording fine-grained trajectory-conditioned motion control. A pivotal component of TrackDiffusion is the instance enhancer, which explicitly ensures inter-frame consistency of multiple objects. generated video sequences by our TrackDiffusion can be used as training data for visual perception models.
  arXiv  Detail & Related papers  (2023-12-01T15:24:38Z)
• Decoupling Dynamic Monocular Videos for Dynamic View Synthesis [50.93409250217699]
  We tackle the challenge of dynamic view synthesis from dynamic monocular videos in an unsupervised fashion. Specifically, we decouple the motion of the dynamic objects into object motion and camera motion, respectively regularized by proposed unsupervised surface consistency and patch-based multi-view constraints.
  arXiv  Detail & Related papers  (2023-04-04T11:25:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.