LivingWorld: Interactive 4D World Generation with Environmental Dynamics

• URL: http://arxiv.org/abs/2604.01641v1
• Date: Thu, 02 Apr 2026 05:38:48 GMT
• Title: LivingWorld: Interactive 4D World Generation with Environmental Dynamics
• Abstract summary: We introduce LivingWorld, an interactive framework for generating 4D worlds with environmental dynamics from a single image.<n>LivingWorld addresses this challenge by progressively constructing a globally coherent motion field as the scene expands.<n>We further represent motion using a compact hash-based motion field, enabling efficient querying and stable propagation of dynamics throughout the scene.
• Score: 8.868060488503847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce LivingWorld, an interactive framework for generating 4D worlds with environmental dynamics from a single image. While recent advances in 3D scene generation enable large-scale environment creation, most approaches focus primarily on reconstructing static geometry, leaving scene-scale environmental dynamics such as clouds, water, or smoke largely unexplored. Modeling such dynamics is challenging because motion must remain coherent across an expanding scene while supporting low-latency user feedback. LivingWorld addresses this challenge by progressively constructing a globally coherent motion field as the scene expands. To maintain global consistency during expansion, we introduce a geometry-aware alignment module that resolves directional and scale ambiguities across views. We further represent motion using a compact hash-based motion field, enabling efficient querying and stable propagation of dynamics throughout the scene. This representation also supports bidirectional motion propagation during rendering, producing long and temporally coherent 4D sequences without relying on expensive video-based refinement. On a single RTX 5090 GPU, generating each new scene expansion step requires 9 seconds, followed by 3 seconds for motion alignment and motion field updates, enabling interactive 4D world generation with globally coherent environmental dynamics. Video demonstrations are available at cvsp-lab.github.io/LivingWorld.

Related papers

• Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory [28.887112448832525]
  We present Matrix-Game 3.5, which advances real-time interactive world generation through three key improvements.<n>First, we propose a unified geometry-aware memory framework, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters.<n>Second, we introduce a static-dynamic disentangled world representation that separately models static scene geometry and dynamic subjects.<n>Third, we develop a two-stage progressive real-time distillation framework that converts a bidirectional diffusion model into a few-step causal generator.
  arXiv  Detail & Related papers  (2026-08-30T17:11:00Z)
• World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video [62.16750462940772]
  We present World from Motion, a method for generating freely renderable dynamic 3D Gaussian representations from monocular videos.<n>Our approach conditions a video model on dense, pixel-aligned renderings that encode appearance, geometry, and 3D scene motion.<n>At test time, we distill the model's generations back into a single consistent, high-quality dynamic 3DGS, improving both novel-view synthesis and the underlying 3D motion.
  arXiv  Detail & Related papers  (2026-07-01T17:41:00Z)
• MotionScale: Reconstructing Appearance, Geometry, and Motion of Dynamic Scenes with Scalable 4D Gaussian Splatting [54.859943475818234]
  MotionScale is a 4D Gaussian Splatting framework that scales efficiently to large scenes and extended sequences.<n>Experiments on challenging real-world benchmarks demonstrate that MotionScale significantly outperforms state-of-the-art methods in both reconstruction quality and temporal stability.
  arXiv  Detail & Related papers  (2026-03-31T06:03:59Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Dexterous World Models [24.21588354488453]
  Dexterous World Model (DWM) is a scene-action-conditioned video diffusion framework.<n>We show how DWM generates temporally coherent videos depicting plausible human-scene interactions.<n>Experiments demonstrate that DWM enables realistic and physically plausible interactions, such as grasping, opening, and moving objects.
  arXiv  Detail & Related papers  (2025-12-19T18:59:51Z)
• WorldReel: 4D Video Generation with Consistent Geometry and Motion Modeling [63.37476802589492]
  We present WorldReel, a 4D video that is native-temporally consistent.<n>WorldReel produces frames together with 4D scene representations, including pointmaps, camera trajectory, and dense flow.<n>We believe that WorldReel brings video generation closer to 4D-consistent world modeling.
  arXiv  Detail & Related papers  (2025-12-08T18:54:12Z)
• SyncMV4D: Synchronized Multi-view Joint Diffusion of Appearance and Motion for Hand-Object Interaction Synthesis [47.61773799705708]
  We introduce SyncMV4D, the first model that jointly generates synchronized multi-view HOI videos and 4D motions.<n>Our method demonstrates superior performance to state-of-the-art alternatives in visual realism, motion plausibility, and multi-view consistency.
  arXiv  Detail & Related papers  (2025-11-24T17:14:19Z)
• Seeing World Dynamics in a Nutshell [132.79736435144403]
  NutWorld is a framework that transforms monocular videos into dynamic 3D representations in a single forward pass.<n>We demonstrate that NutWorld achieves high-fidelity video reconstruction quality while enabling downstream applications in real-time.
  arXiv  Detail & Related papers  (2025-02-05T18:59:52Z)
• 4D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitives [115.67081491747943]
  Dynamic 3D scene representation and novel view synthesis are crucial for enabling AR/VR and metaverse applications.<n>We reformulate the reconstruction of a time-varying 3D scene as approximating its underlying 4D volume.<n>We derive several compact variants that effectively reduce the memory footprint to address its storage bottleneck.
  arXiv  Detail & Related papers  (2024-12-30T05:30:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.