RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation
- URL: http://arxiv.org/abs/2604.19092v1
- Date: Tue, 21 Apr 2026 05:09:56 GMT
- Title: RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation
- Abstract summary: RoboWM-Bench is a manipulation-centric benchmark for embodiment-grounded evaluation of video world models.<n>We evaluate state-of-the-art video world models and find that reliably generating physically executable behaviors remains an open challenge.
- Score: 23.57524297963567
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent advances in large-scale video world models have enabled increasingly realistic future prediction, raising the prospect of leveraging imagined videos for robot learning. However, visual realism does not imply physical plausibility, and behaviors inferred from generated videos may violate dynamics and fail when executed by embodied agents. Existing benchmarks begin to incorporate notions of physical plausibility, but they largely remain perception- or diagnostic-oriented and do not systematically evaluate whether predicted behaviors can be translated into executable actions that complete the intended task. To address this gap, we introduce RoboWM-Bench, a manipulation-centric benchmark for embodiment-grounded evaluation of video world models. RoboWM-Bench converts generated behaviors from both human-hand and robotic manipulation videos into embodied action sequences and validates them through robotic execution. The benchmark spans diverse manipulation scenarios and establishes a unified protocol for consistent and reproducible evaluation. Using RoboWM-Bench, we evaluate state-of-the-art video world models and find that reliably generating physically executable behaviors remains an open challenge. Common failure modes include errors in spatial reasoning, unstable contact prediction, and non-physical deformations. While finetuning on manipulation data yields improvements, physical inconsistencies still persist, suggesting opportunities for more physically grounded video generation for robots.
Related papers
- AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
arXiv Detail & Related papers (2026-08-29T12:55:15Z) - H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models [56.60098789696351]
H2R-Bench is a benchmark for evaluating cross-embodiment human-to-robot manipulation video generation.<n>Each benchmark instance contains a human demonstration video, target embodiment constraints, and source-grounded annotations.<n>We benchmark eleven state-of-the-art video generation models across six manipulation families and two robot embodiments.
arXiv Detail & Related papers (2026-08-13T10:14:33Z) - WALA Learning Executable Latent Actions from Action-Labeled Demonstrations and Action-Free Videos [38.959415159248046]
WALA is a framework for learning executable latent actions from both action-labeled demonstrations and action-free videos.<n>We show that WALA achieves strong performance on RoboTwin, sets a new state-of-the-art result on RoboCasa with 75.2% average success.
arXiv Detail & Related papers (2026-07-13T11:02:04Z) - From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
arXiv Detail & Related papers (2026-05-12T14:15:16Z) - ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignment [31.000965640377128]
ABot-PhysWorld is a 14B Diffusion Transformer model that generates visually realistic, physically plausible, and action-controllable videos.<n>It uses a novel DPO-based post-training framework with decoupled discriminators to suppress unphysical behaviors while preserving visual quality.<n>It achieves new state-of-the-art performance on PBench and EZSbench, surpassing Veo 3.1 and Sora v2 Pro in physical plausibility and trajectory consistency.
arXiv Detail & Related papers (2026-03-24T16:07:09Z) - EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards [47.255807408091755]
Video generative models are increasingly used as world models for robotics.<n>Current video world models lack explicit executability constraints.<n>We introduce Executable Video Alignment (EVA), a reinforcement-learning framework for aligning video world models.
arXiv Detail & Related papers (2026-03-18T15:02:19Z) - Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation [63.04810454548667]
We introduce PhysGen, a scalable continuous and sequential world interaction framework to solve robotic manipulation tasks.<n>By treating the pretrained video model as a proxy for a physics simulator, PhysGen models the dynamic interplay between the external environment and robot actions.<n>We introduce a multimodal continuous representation that unifies video and action into shared physical tokens, bridging the gap between discrete video generation and continuous robotic control.
arXiv Detail & Related papers (2026-02-18T14:58:18Z) - Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
arXiv Detail & Related papers (2025-08-13T13:54:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.