Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising

• URL: http://arxiv.org/abs/2604.26694v2
• Date: Thu, 07 May 2026 11:23:39 GMT
• Title: Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising
• Abstract summary: We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruction) in a single framework.<n>To leverage the strong visual priors of pretrained video diffusion models, X-WAM imagines the future world by predicting multi-view RGB-D videos.<n>Asynchronous Noise Sampling (ANS) to jointly optimize generation quality and action decoding efficiency.
• Score: 22.899605451385824
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose X-WAM, a Unified 4D World Model that unifies real-time robotic action execution and high-fidelity 4D world synthesis (video + 3D reconstruction) in a single framework, addressing the critical limitations of prior unified world models (e.g., UWM) that only model 2D pixel-space and fail to balance action efficiency and world modeling quality. To leverage the strong visual priors of pretrained video diffusion models, X-WAM imagines the future world by predicting multi-view RGB-D videos, and obtains spatial information efficiently through a lightweight structural adaptation: replicating the final few blocks of the pretrained Diffusion Transformer into a dedicated depth prediction branch for the reconstruction of future spatial information. Moreover, we propose Asynchronous Noise Sampling (ANS) to jointly optimize generation quality and action decoding efficiency. ANS applies a specialized asynchronous denoising schedule during inference, which rapidly decodes actions with fewer steps to enable efficient real-time execution, while dedicating the full sequence of steps to generate high-fidelity video. Rather than entirely decoupling the timesteps during training, ANS samples from their joint distribution to align with the inference distribution. Pretrained on over 5,800 hours of robotic data, X-WAM achieves 79.2% and 90.7% average success rate on RoboCasa and RoboTwin 2.0 benchmarks, while producing high-fidelity 4D reconstruction and generation surpassing existing methods in both visual and geometric metrics.

Related papers

• AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing [45.6226084109777]
  We propose AHA-WAM, an Asynchronous Horizon-Adaptive World-Action Model built on a dual Diffusion Transformer (DiT) architecture.<n>We show that AHA-WAM achieves state-of-the-art performance without any robot-data pretraining, attaining 92.80% average success on RoboTwin and 78.3% success across 4 real-world tasks.
  arXiv  Detail & Related papers  (2026-06-08T17:55:18Z)
• Embody4D: A Generalist 4D World Model for Embodied AI [13.488596291534243]
  Embody4D is a dedicated video-to-video world model for embodied scenarios.<n>It is capable of arbitrary novel views from a monocular video.<n>It achieves state-of-the-art performance, serving as a robust world model.
  arXiv  Detail & Related papers  (2026-05-03T09:39:44Z)
• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory [53.39687409541093]
  Matrix-Game 3.0 is a memory-augmented interactive world model designed for 720p real-time longform video generation.<n>We introduce systematic improvements across data, model, and inference.<n> Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model.
  arXiv  Detail & Related papers  (2026-04-10T06:00:09Z)
• Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learning [18.397872306430006]
  Action-conditioned robot world models generate future video frames of the manipulated scene given a robot action sequence.<n>These models are optimized for short-term prediction and break down when deployed autoregressively.<n>We introduce a reinforcement learning scheme that trains the world model on its own autoregressive rollouts.
  arXiv  Detail & Related papers  (2026-03-26T17:36:08Z)
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control [16.562259973551786]
  We introduce DiT4DiT, an end-to-end Video-Action Model that couples a video Diffusion Transformer with an action Diffusion Transformer.<n>Instead of relying on reconstructed future frames, DiT4DiT extracts intermediate denoising features from the video generation process.<n>It achieves state-of-the-art results, reaching average success rates of 98.6% on LIBERO and 50.8% on RoboCasa GR1.
  arXiv  Detail & Related papers  (2026-03-11T06:03:53Z)
• TeleWorld: Towards Dynamic Multimodal Synthesis with a 4D World Model [53.555353366322464]
  We present TeleWorld, a real-time multimodal 4D world modeling framework that unifies video generation, dynamic scene reconstruction, and long-term world memory within a closed-loop system.<n>Our approach achieves seamless integration of dynamic object modeling and static scene representation within a unified 4D framework, advancing world models toward practical, interactive, and computationally accessible synthesis systems.
  arXiv  Detail & Related papers  (2025-12-31T18:31:46Z)
• Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models [79.06910348413861]
  We introduce Diff4Splat, a feed-forward method that synthesizes controllable and explicit 4D scenes from a single image.<n>Given a single input image, a camera trajectory, and an optional text prompt, Diff4Splat directly predicts a deformable 3D Gaussian field that encodes appearance, geometry, and motion.
  arXiv  Detail & Related papers  (2025-11-01T11:16:25Z)
• CVD-STORM: Cross-View Video Diffusion with Spatial-Temporal Reconstruction Model for Autonomous Driving [26.379817613036597]
  CVD-STORM is a cross-view video diffusion model utilizing a spatial-temporal reconstruction Variational Autoencoder (VAE)<n>Our approach first fine-tunes the VAE with an auxiliary 4D reconstruction task, enhancing its ability to encode 3D structures and temporal dynamics.<n> Experimental results demonstrate that our model achieves substantial improvements in both FID and FVD metrics.
  arXiv  Detail & Related papers  (2025-10-09T08:41:58Z)
• 4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming [52.76837132019501]
  We introduce 4DGCPro, a novel hierarchical 4D compression framework.<n>4DGCPro facilitates real-time mobile decoding and high-quality rendering via progressive volumetric video streaming.<n>We present an end-to-end entropy-optimized training scheme.
  arXiv  Detail & Related papers  (2025-09-22T08:38:17Z)
• 4DNeX: Feed-Forward 4D Generative Modeling Made Easy [51.79072580042173]
  We present 4DNeX, the first feed-forward framework for generating 4D (i.e., dynamic 3D) scene representations from a single image.<n>In contrast to existing methods that rely on computationally intensive optimization or require multi-frame video inputs, 4DNeX enables efficient, end-to-end image-to-4D generation.
  arXiv  Detail & Related papers  (2025-08-18T17:59:55Z)
• RAVEN: Rethinking Adversarial Video Generation with Efficient Tri-plane Networks [93.18404922542702]
  We present a novel video generative model designed to address long-term spatial and temporal dependencies. Our approach incorporates a hybrid explicit-implicit tri-plane representation inspired by 3D-aware generative frameworks. Our model synthesizes high-fidelity video clips at a resolution of $256times256$ pixels, with durations extending to more than $5$ seconds at a frame rate of 30 fps.
  arXiv  Detail & Related papers  (2024-01-11T16:48:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.