Action Images: End-to-End Policy Learning via Multiview Video Generation

• URL: http://arxiv.org/abs/2604.06168v2
• Date: Wed, 15 Apr 2026 01:21:52 GMT
• Title: Action Images: End-to-End Policy Learning via Multiview Video Generation
• Abstract summary: We present Action Images, a unified world action model that formulates policy learning as multiview video generation.<n>Our model achieves the strongest zero-shot success rates and improves video-action joint generation quality over prior video-space world models.
• Score: 70.67042168383638
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World action models (WAMs) have emerged as a promising direction for robot policy learning, as they can leverage powerful video backbones to model the future states. However, existing approaches often rely on separate action modules, or use action representations that are not pixel-grounded, making it difficult to fully exploit the pretrained knowledge of video models and limiting transfer across viewpoints and environments. In this work, we present Action Images, a unified world action model that formulates policy learning as multiview video generation. Instead of encoding control as low-dimensional tokens, we translate 7-DoF robot actions into interpretable action images: multi-view action videos that are grounded in 2D pixels and explicitly track robot-arm motion. This pixel-grounded action representation allows the video backbone itself to act as a zero-shot policy, without a separate policy head or action module. Beyond control, the same unified model supports video-action joint generation, action-conditioned video generation, and action labeling under a shared representation. On RLBench and real-world evaluations, our model achieves the strongest zero-shot success rates and improves video-action joint generation quality over prior video-space world models, suggesting that interpretable action images are a promising route to policy learning.

Related papers

• World Tokens: Enhancing Embodied Policies with Training-Time World Modeling [29.442423651761107]
  We introduce World Tokens, an embodied policy architecture built around a World Adapter.<n>World Tokens bridges visual-language understanding, world-dynamics modeling, and action generation.<n>We attains the best reported averages on SIMPLE, substantially improves real-world R1 Pro success over a matched action-only baseline, and generates each action chunk at VLA-level latency.
  arXiv  Detail & Related papers  (2026-08-10T15:30:38Z)
• Masked Visual Actions for Unified World Modeling [96.12988421978463]
  We introduce Masked Visual Actions, a pixel-space control interface that expresses action as a partially revealed trajectory of an arbitrary entity in a video.<n>Revealing robot motion makes the model act as a forward dynamics model that predicts the scene's response to low-level robot actions, while revealing desired object motion makes the same model recover robot behavior consistent with that outcome.
  arXiv  Detail & Related papers  (2026-07-21T17:59:11Z)
• World Action Models: A Survey [100.95337034529263]
  World Action Models (WAMs) are embodied predictive-action models that make a forecast of the future available to action.<n>Recent WAMs repurpose large video generation models, and a parallel line relies on language or vision-language backbones without a video-generation core.<n>This survey gives the field a common account. It first clarifies these boundaries, then organizes existing works through two complementary views.
  arXiv  Detail & Related papers  (2026-06-18T17:05:19Z)
• ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing? [70.07849531480132]
  We propose ImageWAM, a framework that repurposes pretrained image editing models for robot action prediction.<n>ImageWAM does not decode the target frame at inference time; instead, it conditions a flow-matching action expert on the KV caches.<n>It also reduces FLOPs to 1/6 and latency to 1/4 of video-based WAMs.
  arXiv  Detail & Related papers  (2026-06-17T19:25:28Z)
• Turning Video Models into Generalist Robot Policies [35.53955978162271]
  We present a video-to-action policy that combines an action-free video world model with an embodiment-specific inverse dynamics model.<n>Our results show that decoupled video planning plus faithful video-to-action translation is a viable alternative route towards zero-shot, cross-embodiment, and generalizable robot control.
  arXiv  Detail & Related papers  (2026-05-27T01:21:58Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• Unified Video Action Model [47.88377984526902]
  A unified video and action model holds significant promise for robotics, where videos provide rich scene information for action prediction.<n>We introduce the Unified Video Action model (UVA), which jointly optimize video and action predictions to achieve both high accuracy and efficient action inference.<n>Via an extensive set of experiments, we demonstrate that UVA can serve as a general-purpose solution for a wide range of robotics tasks.
  arXiv  Detail & Related papers  (2025-02-28T21:38:17Z)
• Grounding Video Models to Actions through Goal Conditioned Exploration [29.050431676226115]
  We propose a framework that uses trajectory level action generation in combination with video guidance to enable an agent to solve complex tasks.<n>We show how our approach is on par with or even surpasses multiple behavior cloning baselines trained on expert demonstrations.
  arXiv  Detail & Related papers  (2024-11-11T18:43:44Z)
• Learning to Act from Actionless Videos through Dense Correspondences [87.1243107115642]
  We present an approach to construct a video-based robot policy capable of reliably executing diverse tasks across different robots and environments. Our method leverages images as a task-agnostic representation, encoding both the state and action information, and text as a general representation for specifying robot goals. We demonstrate the efficacy of our approach in learning policies on table-top manipulation and navigation tasks.
  arXiv  Detail & Related papers  (2023-10-12T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.