XEWorld: Can Action-Conditioned World Models Generalize to Unseen Robot Embodiments?

• URL: http://arxiv.org/abs/2608.05799v1
• Date: Thu, 06 Aug 2026 09:35:33 GMT
• Title: XEWorld: Can Action-Conditioned World Models Generalize to Unseen Robot Embodiments?
• Abstract summary: Action-conditioned world models are promising learned simulators for robotic manipulation.<n>But evaluating them exclusively on training robots fails to reveal whether they capture physical dynamics or merely visual patterns.<n>We introduce XEWorld, a controlled cross-embodiment testbed for world models that isolates embodiments by evaluating held-out robots within physically identical scenes.
• Score: 19.1327734399886
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Action-conditioned world models are promising learned simulators for robotic manipulation, yet evaluating them exclusively on training robots fails to reveal whether they capture physical dynamics or merely memorize visual patterns. To answer whether a model can faithfully render a robot it has never seen, we introduce XEWorld, a controlled cross-embodiment testbed for world models that isolates embodiments by evaluating held-out robots within physically identical scenes. Our systematic analysis uncovers a shared architectural bottleneck: current models act primarily as 2D visual pattern matchers whose generalization is governed by visual similarity rather than physical kinematic similarity. Driven by this limitation, they struggle to translate abstract numeric joint actions into coherent visual trajectories, and fail to predict dynamic visual changes from static initial observations. Consequently, successfully rendering an unseen embodiment zero-shot strictly requires heavily grounded cues, specifically pixel-space actions and explicit spatial-temporal alignment. Even when bypassing this zero-shot barrier via few-shot adaptation, the forced appearance recovery triggers catastrophic forgetting of seen embodiments. Together, these failures expose a critical inability to apply learned physical dynamics to novel visual appearances, highlighting that achieving true cross-embodiment generalization requires architectural innovations that decouple visual appearance from underlying physical dynamics.

Related papers

• AnyWorld: Factorized Egocentric World Models for Cross-Embodiment Generalization [88.25615187835321]
  We propose AnyWorld, a cross-embodiment world modeling framework.<n>Our model factorizes an interaction into action, camera, and embodiment.<n>We train the model with large-scale human interaction pretraining.
  arXiv  Detail & Related papers  (2026-08-29T12:55:15Z)
• Masked Visual Actions for Unified World Modeling [96.12988421978463]
  We introduce Masked Visual Actions, a pixel-space control interface that expresses action as a partially revealed trajectory of an arbitrary entity in a video.<n>Revealing robot motion makes the model act as a forward dynamics model that predicts the scene's response to low-level robot actions, while revealing desired object motion makes the same model recover robot behavior consistent with that outcome.
  arXiv  Detail & Related papers  (2026-07-21T17:59:11Z)
• Trimming the Long-Tail of Visual World Modeling Evaluation [67.17191772079316]
  We introduce Tailor-Bench, a benchmark that challenges world models to simulate irregular physical interactions.<n>Regular scenarios reflect common tool-task pairs, Unconventional scenarios replace conventional tools with attribute-compatible substitutes, and Impossible scenarios introduce attribute-violating tools.<n>Our experimental results reveal a clear long-tail gap in physical world modeling, indicating limited generalization beyond common interactions.
  arXiv  Detail & Related papers  (2026-06-23T07:43:41Z)
• iMaC: Translating Actions into Motion and Contact Images for Embodied World Models [77.395425755122]
  Embodied world models have emerged as a pivotal paradigm for visual robotic decision-making and interactive environment simulation.<n>This paper proposesiMac (Image as Action Control), a novel unified control paradigm that treats raw visual images as native action representations for embodied world models.
  arXiv  Detail & Related papers  (2026-06-08T17:55:41Z)
• Toward Hardware-Agnostic Quadrupedal World Models via Morphology Conditioning [29.296527466383242]
  We take a step towards a framework for training a generalizable Quadrupedal World Model.<n>We explicitly condition the generative dynamics on the robot's engineering specifications.<n>We introduce, for the first time, a world model that enables zero-shot generalization to new morphologies for locomotion.
  arXiv  Detail & Related papers  (2026-04-09T21:31:24Z)
• HEX: Humanoid-Aligned Experts for Cross-Embodiment Whole-Body Manipulation [74.34984994596813]
  HEX is a state-centric framework for coordinated manipulation on humanoid robots.<n>It incorporates a humanoid-aligned universal state representation for scalable learning across heterogeneous embodiments.<n>It achieves state-of-the-art performance in task success rate and generalization.
  arXiv  Detail & Related papers  (2026-04-09T09:01:43Z)
• ABot-PhysWorld: Interactive World Foundation Model for Robotic Manipulation with Physics Alignment [31.000965640377128]
  ABot-PhysWorld is a 14B Diffusion Transformer model that generates visually realistic, physically plausible, and action-controllable videos.<n>It uses a novel DPO-based post-training framework with decoupled discriminators to suppress unphysical behaviors while preserving visual quality.<n>It achieves new state-of-the-art performance on PBench and EZSbench, surpassing Veo 3.1 and Sora v2 Pro in physical plausibility and trajectory consistency.
  arXiv  Detail & Related papers  (2026-03-24T16:07:09Z)
• Walk through Paintings: Egocentric World Models from Internet Priors [65.30611174953958]
  We present the Egocentric World Model (EgoWM), a simple, architecture-agnostic method that transforms any pretrained video diffusion model into an action-conditioned world model.<n>Rather than training from scratch, we repurpose the rich world priors of Internet-scale video models and inject motor commands through lightweight conditioning layers.<n>Our approach scales naturally across embodiments and action spaces, ranging from 3-DoF mobile robots to 25-DoF humanoids.
  arXiv  Detail & Related papers  (2026-01-21T18:59:32Z)
• Do-Undo: Generating and Reversing Physical Actions in Vision-Language Models [57.71440995598757]
  We introduce the Do-Undo task and benchmark to address a critical gap in vision-language models.<n>Do-Undo requires models to simulate the outcome of a physical action and then accurately reverse it, reflecting true cause-and-effect in the visual world.
  arXiv  Detail & Related papers  (2025-12-15T18:03:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.