ReWorld: Learning Better Representations for World Action Models

• URL: http://arxiv.org/abs/2606.27504v1
• Date: Thu, 25 Jun 2026 19:37:58 GMT
• Title: ReWorld: Learning Better Representations for World Action Models
• Abstract summary: World Action Models (WAMs) model future environment evolution under action conditioning.<n>We propose ReWorld, the first representation learning framework specifically designed for WAMs.<n>We show that ReWorld improves fine-tuned video generation by 23.9% in FVD.
• Score: 41.92917169094194
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World Action Models (WAMs) model future environment evolution under action conditioning, offering a scalable paradigm for autonomous driving. However, existing approaches focus largely on model architecture design, and how a WAM can efficiently learn better world representations for planning remains underexplored. To address this gap, we propose ReWorld, the first representation learning framework specifically designed for autonomous-driving world action models. In WAMs, standard training supervises only the output ends of the generation and planning modules, leaving the intermediate representations that carry world knowledge to be shaped only indirectly, as byproducts of fitting these outputs. The core idea of ReWorld is to treat intermediate representations as direct targets of optimization, shaping them along three complementary dimensions. On the Video DiT responsible for generation, we impose future-predictive supervision on its intermediate representations. On the Action DiT responsible for planning, we first align its intermediate representations cross-modally with the video world representation, then further shape them to be discriminative around safety-critical boundaries via hard-negative supervision. In addition, we systematically analyze the effectiveness of existing representation learning methods in video generation world models, and discuss why their performance is limited on this task. Experiments on nuScenes and NAVSIM show that ReWorld improves fine-tuned video generation by 23.9% in FVD (81.3 to 61.9), raises closed-loop PDMS from 89.1 to 90.4 without any post-training such as RL or post-processing, and accelerates from-scratch convergence by approximately 2x.

Related papers

• Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Driving [51.90209659403234]
  This report presents a unified technical system addressing the two core capabilities of world models for autonomous driving.<n>For world representation, we propose WorldRec, a feed-forward reconstruction architecture driven by sparse scene queries.<n>For world generation, we propose WorldGen, a two-stage training framework of bidirectional pretraining followed by causal fine-tuning.
  arXiv  Detail & Related papers  (2026-05-18T09:46:16Z)
• Learning Vision-Language-Action World Models for Autonomous Driving [15.103497388527943]
  Vision-Language-Action (VLA) models have recently achieved notable progress in end-to-end autonomous driving.<n>VLA models often lack explicit modeling of temporal dynamics and global world consistency.<n>We present VLA-World, a simple yet effective VLA world model that unifies predictive imagination with reflective reasoning.
  arXiv  Detail & Related papers  (2026-04-10T07:38:05Z)
• DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning [44.543763428623976]
  We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning.<n>The proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and latency controllable.
  arXiv  Detail & Related papers  (2026-04-02T08:33:18Z)
• FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment [45.21358158991569]
  VLA models can predict environmental dynamics, known as world modeling.<n>Current approaches face two main issues: 1) The training objective forces models to over-emphasize pixel-level reconstruction, which constrains semantic learning and generalization, and 2) Reliance on predicted future observations during inference often leads to error accumulation.<n>We introduce Future Representation Alignment via Parallel Progressive Expansion (FRAPPE) to address these challenges.
  arXiv  Detail & Related papers  (2026-02-19T11:00:46Z)
• Walk through Paintings: Egocentric World Models from Internet Priors [65.30611174953958]
  We present the Egocentric World Model (EgoWM), a simple, architecture-agnostic method that transforms any pretrained video diffusion model into an action-conditioned world model.<n>Rather than training from scratch, we repurpose the rich world priors of Internet-scale video models and inject motor commands through lightweight conditioning layers.<n>Our approach scales naturally across embodiments and action spaces, ranging from 3-DoF mobile robots to 25-DoF humanoids.
  arXiv  Detail & Related papers  (2026-01-21T18:59:32Z)
• Co-Evolving Latent Action World Models [57.48921576959243]
  Adapting pre-trained video models into controllable world models via latent actions is a promising step towards creating generalist world models.<n>We propose CoLA-World, which for the first time successfully realizes this synergistic paradigm.<n>This unlocks a co-evolution cycle: the world model acts as a knowledgeable tutor, providing gradients to shape a high-quality LAM.
  arXiv  Detail & Related papers  (2025-10-30T12:28:40Z)
• From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction [57.56072009935036]
  We introduce a new driving paradigm named Policy World Model (PWM)<n>PWM integrates world modeling and trajectory planning within a unified architecture.<n>Our method matches or exceeds state-of-the-art approaches that rely on multi-view and multi-modal inputs.
  arXiv  Detail & Related papers  (2025-10-22T14:57:51Z)
• Learning Primitive Embodied World Models: Towards Scalable Robotic Learning [50.32986780156215]
  We propose a novel paradigm for world modeling--Primitive Embodied World Models (PEWM)<n>By restricting video generation to fixed short horizons, our approach enables fine-grained alignment between linguistic concepts and visual representations of robotic actions.<n>Our framework bridges the gap between fine-grained physical interaction and high-level reasoning, paving the way toward scalable, interpretable, and general-purpose embodied intelligence.
  arXiv  Detail & Related papers  (2025-08-28T14:31:48Z)
• RLVR-World: Training World Models with Reinforcement Learning [41.04369775904968]
  We present RLVR-World, a unified framework that leverages reinforcement learning with verifiable rewards.<n>We demonstrate substantial performance gains on both language- and video-based world models across domains, including text games, web navigation, and robot manipulation.
  arXiv  Detail & Related papers  (2025-05-20T05:02:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.