Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning

• URL: http://arxiv.org/abs/2608.24885v1
• Date: Tue, 25 Aug 2026 17:59:49 GMT
• Title: Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning
• Abstract summary: We introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and trajectory alignment.<n>We propose WorldSync, which strengthens action following along three axes: distributional coverage, representational grounding, and intervention-effect alignment.<n>Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement.
• Score: 53.77164209208635
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Action-conditioned world models are increasingly used as learned simulators for policy evaluation and improvement, yet their effectiveness rests on an unverified assumption: generated futures faithfully reflect arbitrary valid actions. Existing benchmarks are typically confined to expert demonstrations, leaving off-expert action following inadequately evaluated. To address this gap, we introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and SE(3) trajectory alignment. Our diagnosis shows that current world models reasonably execute expert actions but struggle with diverse off-expert trajectories, either ignoring the commanded actions or producing visually invalid rollouts. We further propose WorldSync, which strengthens action following along three complementary axes: distributional coverage, representational grounding, and intervention-effect alignment. It broadens the training distribution over action consequences, grounds intermediate video representations in action-induced robot dynamics through an Action-Forcing Expert, and aligns predicted changes under action interventions with the corresponding changes in ground-truth futures. Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement, enabling policies to achieve higher success rates.

Related papers

• WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models [7.861070986808876]
  Post-training VLA policies typically rely on supervised fine-tuning with expensive and potentially unstable real-world exploration.<n>We introduce WISE (World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models), a unified framework that coordinates when and how world-model imagination is used during policy refinement.
  arXiv  Detail & Related papers  (2026-09-03T11:17:57Z)
• On the Capability Separation Between World-Model Policy Learning and Imitated World-Action Models [3.79187263097166]
  We compare a direct behavior-cloning policy, an imitation-trained world-action policy, and a policy optimized with an action-conditioned world model.<n>Future prediction changes the learning factorization but not the unrestricted external policy class or ideal imitation target.
  arXiv  Detail & Related papers  (2026-08-23T03:25:31Z)
• Making Foresight Actionable: Repurposing Representation Alignment in World Action Models [57.23863557252883]
  World Action Models (WAMs) offer a promising route for robot manipulation by using video generation models to model future scene evolution.<n> generating plausible visual futures does not always guarantee the extraction of accurate actions.<n>We propose AGRA, an Action-Grounded Representation Alignment objective.
  arXiv  Detail & Related papers  (2026-06-10T15:31:25Z)
• Feedback World Model Enables Precise Guidance of Diffusion Policy [36.965417653906535]
  We propose feedback world model, a new paradigm that closes the loop between prediction and observation at inference time.<n>We show that our method substantially improves both prediction accuracy and policy performance under distribution shift.<n>In particular, it reduces world model prediction error by up to 76.4% and improves out-of-distribution (OOD) success rate by 30%.
  arXiv  Detail & Related papers  (2026-05-15T07:52:13Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Reshaping Action Error Distributions for Reliable Vision-Language-Action Models [69.38615670891038]
  In robotic manipulation, vision-language-action (VLA) models have emerged as a promising paradigm for learning generalizable and scalable robot policies.<n>We focus on continuous-action VLA models and move beyond conventional MSE-based regression by reshaping action error distributions during training.<n>We evaluate our approaches across standard, few-shot, and noisy settings on multiple representative VLA architectures.
  arXiv  Detail & Related papers  (2026-02-04T05:37:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.