GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

• URL: http://arxiv.org/abs/2607.02642v1
• Date: Thu, 02 Jul 2026 17:02:43 GMT
• Title: GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation
• Abstract summary: WMBench is a benchmark constructed from real-robot teleoperation data and matched policy rollouts.<n>We analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions.<n>Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism.
• Score: 73.02743604295246
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in \textit{GigaWorld-1}, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.

Related papers

• RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation [52.855830378221775]
  Video world models are emerging as a scalable alternative for evaluating generalist robot policies.<n>We introduce RoboWorld, an automated evaluation pipeline that pairs a fast video world model with a task-progress-aware vision-language model scoring.
  arXiv  Detail & Related papers  (2026-07-01T15:22:41Z)
• World Model for Robot Learning: A Comprehensive Survey [168.5691568303064]
  This survey systematically reviews the rapidly growing literature on world models for robot learning.<n>We examine how world models are coupled with robot policies, how they serve as learned simulators for reinforcement learning and evaluation, and how robotic video world models have progressed from imagination-based generation to controllable, structured, and foundation-scale formulations.<n>Overall, this survey systematically reviews the rapidly growing literature on world models for robot learning, clarifies key paradigms and applications, and highlights major challenges and future directions for predictive modeling in embodied agents.
  arXiv  Detail & Related papers  (2026-04-30T14:35:31Z)
• Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training [54.896907620476675]
  We propose a post-training framework that uses a learned world model as a reusable corrective substrate for failure-targeted policy improvement.<n>Hi-WM caches intermediate states and supports rollback and branching, allowing a single failure state to be reused for multiple corrective continuations.<n>We evaluate Hi-WM on three real-world manipulation tasks spanning both rigid and deformable object interaction, and on two policy backbones.
  arXiv  Detail & Related papers  (2026-04-23T14:42:54Z)
• Interactive World Simulator for Robot Policy Training and Evaluation [21.481187472784047]
  We present Interactive World Simulator, a framework for building interactive world models from a robot interaction dataset.<n>In our experiments, the learned world models produce interaction-consistent pixel-level predictions.<n>We find that policies trained on world-model-generated data perform comparably to those trained on the same amount of real-world data.
  arXiv  Detail & Related papers  (2026-03-09T16:13:32Z)
• RobotArena $\infty$: Scalable Robot Benchmarking via Real-to-Sim Translation [47.79800816696372]
  Real-world testing of manipulation policies is labor-intensive at scale, and difficult to reproduce.<n>Existing simulation benchmarks are similarly limited, as they train and test policies within the same synthetic domains.<n>In this paper, we introduce a new benchmarking framework that overcomes these challenges by shifting VLA evaluation into large-scale simulated augmented environments.
  arXiv  Detail & Related papers  (2025-10-27T17:41:38Z)
• Ctrl-World: A Controllable Generative World Model for Robot Manipulation [53.71061464925014]
  Generalist robot policies can perform a wide range of manipulation skills.<n> evaluating and improving their ability with unfamiliar objects and instructions remains a significant challenge.<n>World models offer a promising, scalable alternative by enabling policies to rollout within imagination space.
  arXiv  Detail & Related papers  (2025-10-11T09:13:10Z)
• WorldGym: World Model as An Environment for Policy Evaluation [41.204900701616914]
  WorldGym is an autoregressive, action-conditioned video generation model which serves as a proxy to real world environments.<n> Policies are evaluated via Monte Carlo rollouts in the world model, with a vision-language model providing rewards.<n>We show that WorldGym is able to preserve relative policy rankings across different policy versions, sizes, and training checkpoints.
  arXiv  Detail & Related papers  (2025-05-31T15:51:56Z)
• Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics [50.191655141020505]
  This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer.<n>By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.
  arXiv  Detail & Related papers  (2025-01-17T10:39:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.