DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation

• URL: http://arxiv.org/abs/2608.22067v4
• Date: Mon, 31 Aug 2026 05:42:58 GMT
• Title: DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
• Abstract summary: We argue that video generation is an unnecessary objective for world-action modeling.<n>For robotic manipulation, the goal of a world model is not to reproduce how the world looks at every intermediate moment, but to predict the state that the world will reach after an action is executed.<n>We propose DELE-w0.5, which infers robot actions from predicted future states without relying on video generation.
• Score: 11.665741844323984
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: World-Action Models (WAMs) build robot control on video-generation backbones, which jointly predict dense future visual trajectories and robot actions. We argue that video generation is an unnecessary intermediate objective for world-action modeling. For robotic manipulation, the goal of a world model is not to reproduce how the world looks at every intermediate moment, but to predict the state that the world will reach after an action is executed. The intermediate frames only describe the visual transition between physical states, which consumes substantial model capacity and computation, but do not directly specify the physical outcome that the robot action is intended to produce. In this paper, we propose DELE-w0.5, which infers robot actions from predicted future states without relying on video generation. Concretely, DELE-w0.5 infers the action sequence from its corresponding compact future latent state. The future latent state captures the action-relevant physical outcome of robot interaction and serves as an explicit bridge between world modeling and action generation. The core design principle of DELE-w0.5 is to model how the physical world changes under robot actions, rather than how its visual appearance evolves frame by frame. This formulation removes the high-dimensional visual redundancy introduced by dense video representations, and it therefore enables cheaper training and low-latency inference. Across 640 real-robot trials on four long-horizon manipulation tasks, our DELE-w0.5 achieves the best performance among all compared policies, attaining 62.5% overall full-task success and 81.3% macro ordered-stage progress. It outperforms the strongest baseline by 32.5 percentage points in full-task success and 20.1 percentage points in macro progress.

Related papers

• ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training [93.39352617578857]
  We introduce ZimaBlue, a framework for learning generalizable World Action Models from large-scale video.<n>ZimaBlue follows a three-stage training curriculum, first performing causal embodied video pre-training on large-scale human and robot egocentric videos.<n>ZimaBlue further delivers strong performance across multiple benchmarks, with particularly pronounced gains on unseen tasks.
  arXiv  Detail & Related papers  (2026-08-31T18:09:52Z)
• H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models [56.60098789696351]
  H2R-Bench is a benchmark for evaluating cross-embodiment human-to-robot manipulation video generation.<n>Each benchmark instance contains a human demonstration video, target embodiment constraints, and source-grounded annotations.<n>We benchmark eleven state-of-the-art video generation models across six manipulation families and two robot embodiments.
  arXiv  Detail & Related papers  (2026-08-13T10:14:33Z)
• Robot-Factored World Models via Robot Rendering [25.12876640584098]
  Action-conditioned video world models predict future observations from an initial observation and an action signal.<n>We propose robot-factored world models, which move two robot-specific factors outside the world model.<n>Our experiments show that the rendered interface outperforms vector-conditioned baselines and generalizes to unseen robot embodiments inference.
  arXiv  Detail & Related papers  (2026-07-24T17:59:57Z)
• Masked Visual Actions for Unified World Modeling [96.12988421978463]
  We introduce Masked Visual Actions, a pixel-space control interface that expresses action as a partially revealed trajectory of an arbitrary entity in a video.<n>Revealing robot motion makes the model act as a forward dynamics model that predicts the scene's response to low-level robot actions, while revealing desired object motion makes the same model recover robot behavior consistent with that outcome.
  arXiv  Detail & Related papers  (2026-07-21T17:59:11Z)
• LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies [31.88655699486955]
  We present LaWAM, a latent-action-conditioned Latent World Model (LaWM)<n>LaWAM exposes predictive dynamics to robot policies through compact latent visual subgoals instead of reconstructed future video.<n>LaWAM runs in 187 ms per action-chunk prediction and achieves up to 24x lower wall-clock latency than pixel-space WAMs.
  arXiv  Detail & Related papers  (2026-06-14T12:06:58Z)
• OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics [6.134835623350618]
  We present a precise action-conditioned video world model that generalizes across different robot embodiments and enables robot policy evaluation.<n>Our model achieves significant improvement on action following, appearance quality, and motion consistency, compared to existing baselines.<n>We further deploy OSCAR to evaluate robot policies from RoboArena.
  arXiv  Detail & Related papers  (2026-06-03T05:16:41Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control [16.562259973551786]
  We introduce DiT4DiT, an end-to-end Video-Action Model that couples a video Diffusion Transformer with an action Diffusion Transformer.<n>Instead of relying on reconstructed future frames, DiT4DiT extracts intermediate denoising features from the video generation process.<n>It achieves state-of-the-art results, reaching average success rates of 98.6% on LIBERO and 50.8% on RoboCasa GR1.
  arXiv  Detail & Related papers  (2026-03-11T06:03:53Z)
• Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation [63.04810454548667]
  We introduce PhysGen, a scalable continuous and sequential world interaction framework to solve robotic manipulation tasks.<n>By treating the pretrained video model as a proxy for a physics simulator, PhysGen models the dynamic interplay between the external environment and robot actions.<n>We introduce a multimodal continuous representation that unifies video and action into shared physical tokens, bridging the gap between discrete video generation and continuous robotic control.
  arXiv  Detail & Related papers  (2026-02-18T14:58:18Z)
• Large Video Planner Enables Generalizable Robot Control [117.49024534548319]
  General-purpose robots require decision-making models that generalize across diverse tasks and environments.<n>Recent works build robot foundation models by extending multimodal large language models (LMs) with action outputs, creating vision--action (VLA) systems.<n>We explore an alternative paradigm of using large-scale video pretraining as a primary modality for building robot foundation models.
  arXiv  Detail & Related papers  (2025-12-17T18:35:54Z)
• MiVLA: Towards Generalizable Vision-Language-Action Model with Human-Robot Mutual Imitation Pre-training [102.850162490626]
  We propose MiVLA, a vision-language-action model empowered by human-robot mutual imitation pre-training.<n>We show that MiVLA achieves strong improved generalization capability, outperforming state-of-the-art VLAs.
  arXiv  Detail & Related papers  (2025-12-17T12:59:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.