When to Trust Imagination: Adaptive Action Execution for World Action Models

• URL: http://arxiv.org/abs/2605.06222v2
• Date: Sat, 09 May 2026 10:59:29 GMT
• Title: When to Trust Imagination: Adaptive Action Execution for World Action Models
• Abstract summary: World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation by jointly predicting future visual observations and future actions.<n>Current WAMs execute a fixed number of predicted actions after each model inference, leaving the robot blind to whether the imagined future remains consistent with the actual physical rollout.<n>We formulate adaptive WAM execution as a future-reality verification problem: the robot should execute longer when the WAM-predicted future remains reliable, and replan earlier when reality deviates from imagination.
• Score: 42.51856318901667
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World Action Models (WAMs) have recently emerged as a promising paradigm for robotic manipulation by jointly predicting future visual observations and future actions. However, current WAMs typically execute a fixed number of predicted actions after each model inference, leaving the robot blind to whether the imagined future remains consistent with the actual physical rollout. In this work, we formulate adaptive WAM execution as a future-reality verification problem: the robot should execute longer when the WAM-predicted future remains reliable, and replan earlier when reality deviates from imagination. To this end, we propose Future Forward Dynamics Causal Attention (FFDC), a lightweight verifier that jointly reasons over predicted future actions, predicted visual dynamics, real observations, and language instructions to estimate whether the remaining action rollout can still be trusted. FFDC enables adaptive action chunk sizes as an emergent consequence of prediction-observation consistency, preserving the efficiency of long-horizon execution while restoring responsiveness in contact-rich or difficult phases. We further introduce Mixture-of-Horizon Training to improve long-horizon trajectory coverage for adaptive execution. Experiments on the RoboTwin benchmark and in the real world demonstrate that our method achieves a strong robustness-efficiency trade-off: on RoboTwin, it reduces WAM forward passes by 69.10% and execution time by 34.02%, while improving success rate by 2.54% over the short-chunk baseline; in real-world experiments, it improves success rate by 35%.

Related papers

• Do Robotic World Models Really Follow Actions? Diagnosing and Aligning Action-Conditioned Generation for Policy Learning [53.77164209208635]
  We introduce WorldEcho, which probes action following over a broader action distribution using visual integrity and trajectory alignment.<n>We propose WorldSync, which strengthens action following along three axes: distributional coverage, representational grounding, and intervention-effect alignment.<n>Experiments on RoboTwin benchmarks and real-robot tasks show that WorldSync improves WorldEcho metrics and serves as a more reliable simulator for iterative policy improvement.
  arXiv  Detail & Related papers  (2026-08-25T17:59:49Z)
• Latent Action as Intention Enables Efficient Future Imagination for World Action Models [20.031922437200524]
  We introduce **LAWA**, a WAM architecture that uses compact latent actions as an operational representation of future intentions.<n>On RoboCasa, LAWA achieves state-of-the-art average success rates of 65.6% and 80.8% in the few-shot and full data settings.<n>It also preserves the performance level of the matched Joint-WAM variant while requiring 42.9% lower inference latency.
  arXiv  Detail & Related papers  (2026-08-25T17:59:03Z)
• Foresight Without Seeing: Latent Futures for World Action Models [31.423405133926334]
  We propose ForeWAM, a dynamics-conditioned direct-policy WAM that provides predictive context for action generation without decoding future videos.<n>We also introduce dynamics supervised by a frozen latent action teacher, encouraging the implicit future states to capture interaction-induced transitions such as object motion, contact changes, and task progress.<n>Without embodied robot data pretraining, the standard and accelerated variants of ForeWAM achieve average success rates of 96.7% and 96.9% on LIBERO, respectively.
  arXiv  Detail & Related papers  (2026-08-12T03:27:43Z)
• FBFM: A Training-Free Asynchronous Feedback Mechanism for Flow-Matching in World-Action Models Execution [50.18922379062543]
  We present Feedback Flow Matching (FBFM), a training-free inference mechanism that pushes re-grounding inside actively generated chunks.<n>Being training-free, the mechanism improves responsiveness to unexpected events and suppresses drift in long-horizon tasks.<n>We evaluate FBFM on both a joint-generation WAM and a stage-wise WAM.
  arXiv  Detail & Related papers  (2026-07-31T10:11:09Z)
• ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts [24.959404195743744]
  Training-Distribution Hallucination is a recurring phenomenon in which futures conditioned on visually shifted observations hallucinate training-domain content rather than remain faithful to the current scene.<n>We propose Semantic-Temporal WAM (ST-WAM) to improve action robustness by using DINOv3 as a shared semantic representation for future prediction and history retrieval while retaining fine-grained VAE dynamics.
  arXiv  Detail & Related papers  (2026-07-31T03:44:56Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Models [28.165928090888986]
  World Action Models (WAMs) enable decision-making through imagined rollouts by predicting future observations and actions.<n>We identify action-state consistency, the alignment between predicted actions and induced state transitions, as a missing reliability axis for WAMs.<n>We introduce a value-free consensus strategy for test-time selection, which ranks candidate rollouts by agreement among predicted futures.
  arXiv  Detail & Related papers  (2026-05-08T09:44:43Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• GigaWorld-Policy: An Efficient Action-Centered World--Action Model [50.107640832046464]
  GigaWorld-Policy is an action-centered WAM that learns 2D pixel-action dynamics while enabling efficient action decoding, with optional video generation.<n> Experimental results on real-world robotic platforms show that GigaWorld-Policy runs 9x faster than the leading WAM baseline, Motus.
  arXiv  Detail & Related papers  (2026-03-18T00:52:02Z)
• See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation [59.07792608884117]
  We introduce See, Plan, Rewind (SPR), a progress-aware vision-language-action framework.<n>SPR operates through a continuous core cycle, Seeing the current state and upcoming milestone, Planning a trajectory towards the next 2D waypoint, and Rewinding to a recoverable state upon failure.<n>SPR achieves state-of-the-art robustness with the smallest performance drop, surpassing OpenVLA-OFT and UniVLA.
  arXiv  Detail & Related papers  (2026-03-10T07:22:51Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)
• FUTURE-VLA: Forecasting Unified Trajectories Under Real-time Execution [27.399648455932397]
  FUTURE-VLA is a unified architecture that reformulates long-horizon control and future forecasting as a monolithic sequence-generation task.<n> FUTURE-VLA achieves success rates of 99.2% on LIBERO, 75.4% on RoboTwin, and 78.0% on a real-world Piper platform.
  arXiv  Detail & Related papers  (2026-02-05T14:27:43Z)
• MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Anticipation [17.4088244981231]
  Long-term dense action anticipation is challenging since it requires predicting actions and their durations several minutes into the future.<n>We propose a novel MANTA (MAmba for ANTicipation) network to enable effective long-term temporal modelling.<n>Our approach achieves state-of-the-art results on three datasets - Breakfast, 50Salads, and Assembly101.
  arXiv  Detail & Related papers  (2025-01-15T14:46:44Z)
• DiffAnt: Diffusion Models for Action Anticipation [12.022815981853071]
  Anticipating future actions is inherently uncertain. Given an observed video segment containing ongoing actions, multiple subsequent actions can plausibly follow. In this work, we rethink action anticipation from a generative view, employing diffusion models to capture different possible future actions. Our code and trained models will be published on GitHub.
  arXiv  Detail & Related papers  (2023-11-27T16:40:09Z)
• Future Transformer for Long-term Action Anticipation [33.771374384674836]
  We propose an end-to-end attention model for action anticipation, dubbed Future Transformer (FUTR) Unlike the previous autoregressive models, the proposed method learns to predict the whole sequence of future actions in parallel decoding. We evaluate our method on two standard benchmarks for long-term action anticipation, Breakfast and 50 Salads, achieving state-of-the-art results.
  arXiv  Detail & Related papers  (2022-05-27T14:47:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.