Lifting Embodied World Models for Planning and Control

• URL: http://arxiv.org/abs/2604.26182v1
• Date: Tue, 28 Apr 2026 23:59:19 GMT
• Title: Lifting Embodied World Models for Planning and Control
• Abstract summary: We train a lightweight policy that maps high-level actions to sequences of low-level joint actions.<n>We instantiate this framework for a human-like embodiment, defining the high-level action space as a small set of 2D waypoints.<n>We show that the lifted world model substantially outperforms searching directly in low-level joint space.
• Score: 59.09016913513998
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World models of embodied agents predict future observations conditioned on an action taken by the agent. For complex embodiments, action spaces are high-dimensional and difficult to specify: for example, precisely controlling a human agent requires specifying the motion of each joint. This makes the world model hard to control and expensive to plan with as search-based methods like CEM scale poorly with action dimensionality. To address this issue, we train a lightweight policy that maps high-level actions to sequences of low-level joint actions. Composing this policy with the frozen world model produces a lifted world model that predicts a sequence of future observations from a single high-level action. We instantiate this framework for a human-like embodiment, defining the high-level action space as a small set of 2D waypoints annotated on the current observation frame, each specifying a near-term goal position for a leaf joint (pelvis, head, hands). Waypoints are low-dimensional, visually interpretable, and easy to specify manually or to search over. We show that the lifted world model substantially outperforms searching directly in low-level joint space ($3.8\times$ lower mean joint error to the goal pose), while remaining more compute-efficient and generalizing to environments unseen by the policy.

Related papers

• GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation [54.305781492241785]
  We present a framework for learning visuomotor manipulation policies that generalize beyond the training distribution.<n>GHOST factorizes control into (i) a high-level policy that predicts the next sub-goal as a distribution over 3D end-effector poses from multi-view RGB-D observations.<n>Across a suite of tasks, this hierarchical factorization consistently improves performance and robustness compared to a flat Diffusion Policy.
  arXiv  Detail & Related papers  (2026-06-08T18:08:01Z)
• Hierarchical Planning with Latent World Models [49.82095442483551]
  hierarchical planning achieves higher success while requiring up to 4x less planning-time compute.<n>We demonstrate that this hierarchical approach enables zero-shot control on real-world non-greedy robotic tasks.
  arXiv  Detail & Related papers  (2026-04-03T17:32:36Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Scaling World Model for Hierarchical Manipulation Policies [61.736772957803026]
  Vision-Language-Action (VLA) models are promising for generalist robot manipulation but brittle in out-of-distribution settings.<n>We introduce a hierarchical Vision-Language-Action framework our that leverages the generalization of large-scale pre-trained world model.<n>We validate both visual goal synthesis and our hierarchical VLA policies in massive out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2026-02-11T16:12:33Z)
• Act2Goal: From World Model To General Goal-conditioned Policy [14.222177107215648]
  Act2Goal is a goal-conditioned manipulation policy that integrates a goal-conditioned visual world model with multi-scale temporal control.<n>We show that Act2Goal improves success rates from 30% to 90% on challenging out-of-distribution tasks within minutes of autonomous interaction.
  arXiv  Detail & Related papers  (2025-12-29T15:28:42Z)
• Ctrl-World: A Controllable Generative World Model for Robot Manipulation [53.71061464925014]
  Generalist robot policies can perform a wide range of manipulation skills.<n> evaluating and improving their ability with unfamiliar objects and instructions remains a significant challenge.<n>World models offer a promising, scalable alternative by enabling policies to rollout within imagination space.
  arXiv  Detail & Related papers  (2025-10-11T09:13:10Z)
• WorldPrediction: A Benchmark for High-level World Modeling and Long-horizon Procedural Planning [52.36434784963598]
  We introduce WorldPrediction, a video-based benchmark for evaluating world modeling and procedural planning capabilities of different AI models.<n>We show that current frontier models barely achieve 57% accuracy on WorldPrediction-WM and 38% on WorldPrediction-PP whereas humans are able to solve both tasks perfectly.
  arXiv  Detail & Related papers  (2025-06-04T18:22:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.