From World Models to World Action Models: A Concise Tutorial for Robotics

• URL: http://arxiv.org/abs/2607.00836v3
• Date: Mon, 06 Jul 2026 10:04:32 GMT
• Title: From World Models to World Action Models: A Concise Tutorial for Robotics
• Abstract summary: This tutorial presents a design-space view of world models as action-conditioned predictive models.<n>The goal of this tutorial is to clarify the conceptual scope of world (action) models and provide a structured taxonomy for embodied prediction and control.
• Score: 3.9330868250536457
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World models are increasingly used in embodied intelligence and generative simulation, yet their scope remains ambiguous across communities. This tutorial presents a design-space view of world models as action-conditioned predictive models that estimate the future evolution of task-relevant observations or states. We categorize existing methods into observation-space and state-space world models, comparing their trade-offs in visual fidelity, spatial structure, physical interpretability, and control usability. We further introduce world action models, which connect predicted futures with executable robot actions, and summarize four representative paradigms: imagine-then-execute, video-feature-conditioned action prediction, joint video-action modeling, and auxiliary video prediction for policy learning. The goal of this tutorial is to clarify the conceptual scope of world (action) models and provide a structured taxonomy for embodied prediction and control.

Related papers

• World-Action Models for Robot Learning and Control: A Survey [41.45329550202786]
  Recent advances in world models, video generation, and Vision-Language-Action (VLA) policies have motivated the development of World-Action Models (WAMs)<n>This survey provides a robotics-oriented review of WAMs.<n>We clarify their scope relative to conventional world models, model-based reinforcement learning, action-conditioned video generation, and reactive VLA policies, and organize existing methods through a unified taxonomy.<n>We discuss key challenges in action alignment, world-action factorization, spatial and multi-view consistency, long-horizon memory, neural simulation for closed-loop policy learning, and efficient inference
  arXiv  Detail & Related papers  (2026-09-13T16:35:56Z)
• Predicting Consequences and Reinforcing Navigation Policies with Latent World Models [31.865100130203317]
  We propose a compatibility prediction Latent World Model (LWM) for robot navigation.<n>Our key insight is that spatial proximity correlates with latent feature similarity, enabling action consequences to be evaluated directly in latent space.<n>Our approach significantly outperforms prior world model and imitation learning methods in prediction accuracy, policy learning, and real-world navigation performance.
  arXiv  Detail & Related papers  (2026-08-23T11:58:42Z)
• World Models for Robotic Manipulation: A Survey [30.864774708923147]
  We survey world models for robotic manipulation through three questions.<n>We operationally define a world model as an action-conditioned predictive system.<n>We characterize infrastructure roles including synthetic experience generation, candidate filtering, search-based evaluation, learned environments, and outcome verification.
  arXiv  Detail & Related papers  (2026-05-27T05:32:17Z)
• Nano World Models: A Minimalist Implementation of Future Video Prediction [52.36220976002044]
  Nano World Models is a minimalist for future video prediction centered around diffusion forcing.<n>By releasing code, configurations, evaluation scripts, and pretrained checkpoints, Nano World Models aims to provide a compact yet experimental substrate for open, reproducible, and scientific world-model research.
  arXiv  Detail & Related papers  (2026-05-17T22:46:44Z)
• World Action Models: The Next Frontier in Embodied AI [123.5787299299832]
  Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning.<n>They learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention.<n>A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline.
  arXiv  Detail & Related papers  (2026-05-12T13:10:52Z)
• World Model for Robot Learning: A Comprehensive Survey [168.5691568303064]
  This survey systematically reviews the rapidly growing literature on world models for robot learning.<n>We examine how world models are coupled with robot policies, how they serve as learned simulators for reinforcement learning and evaluation, and how robotic video world models have progressed from imagination-based generation to controllable, structured, and foundation-scale formulations.<n>Overall, this survey systematically reviews the rapidly growing literature on world models for robot learning, clarifies key paradigms and applications, and highlights major challenges and future directions for predictive modeling in embodied agents.
  arXiv  Detail & Related papers  (2026-04-30T14:35:31Z)
• Causal World Modeling for Robot Control [56.31803788587547]
  Video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics.<n>We introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously.<n>We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations.
  arXiv  Detail & Related papers  (2026-01-29T17:07:43Z)
• A Step Toward World Models: A Survey on Robotic Manipulation [58.8419978790227]
  We look at approaches that exhibit the core capabilities of world models through a review of methods in robotic manipulation.<n>We analyze their roles across perception, prediction, and control, identify key challenges and solutions, and distill the core components, capabilities, and functions that a fully realized world model should possess.
  arXiv  Detail & Related papers  (2025-10-31T00:57:24Z)
• Semantic World Models [11.950909271505923]
  Planning with world models offers a powerful paradigm for robotic control.<n>This paper posits that instead of reconstructing future frames as pixels, world models only need to predict task-relevant semantic information about the future.
  arXiv  Detail & Related papers  (2025-10-22T17:53:45Z)
• From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction [57.56072009935036]
  We introduce a new driving paradigm named Policy World Model (PWM)<n>PWM integrates world modeling and trajectory planning within a unified architecture.<n>Our method matches or exceeds state-of-the-art approaches that rely on multi-view and multi-modal inputs.
  arXiv  Detail & Related papers  (2025-10-22T14:57:51Z)
• Object-Centric World Model for Language-Guided Manipulation [4.008780119020479]
  A world model is essential for an agent to predict the future and plan in domains such as autonomous driving and robotics.<n>We propose a world model leveraging object-centric representation space using slot attention, guided by language instructions.<n>Our model perceives the current state as an object-centric representation and predicts future states in this representation space conditioned on natural language instructions.
  arXiv  Detail & Related papers  (2025-03-08T11:17:37Z)
• Predictive World Models from Real-World Partial Observations [66.80340484148931]
  We present a framework for learning a probabilistic predictive world model for real-world road environments. While prior methods require complete states as ground truth for learning, we present a novel sequential training method to allow HVAEs to learn to predict complete states from partially observed states only.
  arXiv  Detail & Related papers  (2023-01-12T02:07:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.