GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

• URL: http://arxiv.org/abs/2609.05588v1
• Date: Fri, 04 Sep 2026 17:16:48 GMT
• Title: GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
• Abstract summary: We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all from scratch on manipulation data.<n>It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM)<n>The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action.
• Score: 63.853929983870955
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World-action models (WAM) predict future states to guide robot actions, enabling learning from both action-free video and action-labeled interaction. Most inherit pretrained video generators, leaving WAM pretraining and scaling underexplored. We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all initialized from scratch on manipulation data. It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM). CoAE retains action- and instruction-relevant information under aggressive compression, while SVP produces a complete future state in one differentiable pass, so visual planning and inverse dynamics can be pretrained separately on complementary data. The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action. We evaluate pretrained checkpoints directly, without per-task fine-tuning, on 100 tasks across 20 manipulation skill groups with held-out scenes, backgrounds, lighting, and object instances. Scaling co-training data from 300 to 30,000 hours raises success from 17.1% to 44.1% on G1-OP and from 13.4% to 31.1% on G2-90D; despite comprising less than 2% of the co-training data, G2-90D improves by 17.7 points, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage strongly correlates with zero-shot out-of-distribution (OOD) success (Pearson r=0.80; Spearman rho=0.85). Under the same protocol, the model grounds object, color, shape, and position references in at least 90% of trials and follows explicit instructions even when they conflict with an already-committed behavior or a conventional scene association.

Related papers

• DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation [9.154987050049268]
  We introduce DECOWAM, a whole-body world-action model that separates camera ego-motion from base and arm actions.<n>On a fixed replay protocol, DECOWAM improved both future-video and action prediction over FastWAM, reducing action MSE by 21.7% with 25.95M trainable adaptation parameters.<n>These results show that embodiment-aware factorization can support parameter-efficient joint visual prediction and whole-body control under moving viewpoints.
  arXiv  Detail & Related papers  (2026-08-20T14:44:11Z)
• HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation [16.161933557281447]
  HiTac-WAM is a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution.<n>For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates.<n>For execution, the selected tactile forecast is retained as a reference.
  arXiv  Detail & Related papers  (2026-08-20T02:28:25Z)
• Hydra-0: Action Flow for Generalist World Modeling and Control [64.88849745875753]
  Hydra-0 is a world model conditioned on action flow, which represents robot actions as pixel motion.<n>Our best configuration achieves 90.4% lower robot-motion error and 60.2% lower object-motion error.<n>A trained action head maps the resulting latent features to executable actions without requiring task-specific expert robot demonstrations.
  arXiv  Detail & Related papers  (2026-08-18T17:59:30Z)
• G0.5: One Autoregressive Stream for Robot Reasoning and Action [43.43377463558725]
  We introduce G0.5, a pretrained autoregressive VLA in which a single transformer decoder emits reasoning and action tokens under a single objective.<n>Because reasoning and action share a single set of weights, the pretrained VLM's capabilities carry over to physical behavior.<n>G0.5 surpasses state-of-the-art models across 7 independent regimes.
  arXiv  Detail & Related papers  (2026-08-12T07:26:47Z)
• SABER: A Scalable Action-Based Embodied Dataset for Real-World VLA Adaptation [0.0]
  SABER is a high-fidelity retail robotics action dataset built from over 100 hours of natural in-store capture.<n>It contains 44.8K training samples across three action representation streams.<n>It yields a mean success rate of 29.3% across ten retail manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-10T15:51:01Z)
• Disentangled Robot Learning via Separate Forward and Inverse Dynamics Pretraining [28.30092786035367]
  DeFI is a novel framework that Decouples visual Forward and Inverse dynamics pretraining to exploit respective data sources.<n>We introduce the General Forward Dynamics Model (GFDM), pretrained on diverse human and robot videos for future prediction, and the General Inverse Dynamics Model (GIDM), trained via self-supervised learning to infer latent actions from unlabeled video transitions.<n>Experiments on CALVIN ABC-D and SimplerEnv demonstrate state-of-the-art performance, with DeFI achieving an average task length of 4.51 for CALVIN, 51.2% success rate on SimplerEnv-Frac
  arXiv  Detail & Related papers  (2026-03-27T17:20:10Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• World Action Models are Zero-shot Policies [111.91938055103633]
  We introduce DreamZero, a World Action Model (WAM) built upon a pretrained video diffusion backbone.<n>By jointly modeling video and action, DreamZero learns diverse skills effectively from heterogeneous robot data.<n>We demonstrate two forms of cross-embodiment transfer: video-only demonstrations from other robots or humans yield a relative improvement of over 42% on unseen task performance.
  arXiv  Detail & Related papers  (2026-02-17T15:04:02Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• EgoAgent: A Joint Predictive Agent Model in Egocentric Worlds [119.02266432167085]
  We propose EgoAgent, a unified agent model that simultaneously learns to represent, predict, and act within a single transformer.<n>EgoAgent explicitly models the causal and temporal dependencies among these abilities by formulating the task as an interleaved sequence of states and actions.<n> Comprehensive evaluations of EgoAgent on representative tasks such as image classification, egocentric future state prediction, and 3D human motion prediction demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2025-02-09T11:28:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.