Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models

• URL: http://arxiv.org/abs/2609.06578v1
• Date: Sun, 06 Sep 2026 12:46:20 GMT
• Title: Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models
• Abstract summary: World Action Models (WAMs) extend Vision-Language-Action (VLA) models by incorporating future visual dynamics into action generation.<n>Existing WAMs often utilize imagined futures with limited adaptation to evolving execution progress, potentially introducing distracting or unreliable predictive cues.<n>We propose ProWAM, a Progress-Conditioned World Action Model that introduces execution progress as an explicit intermediate representation for imagination utilization.
• Score: 72.02240114225107
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World Action Models (WAMs) extend Vision-Language-Action (VLA) models by incorporating future visual dynamics into action generation. However, existing WAMs often utilize imagined futures with limited adaptation to evolving execution progress, potentially introducing distracting or unreliable predictive cues. This limitation arises from two empirically identified forms of non-uniformity in future utility: (i) at the inter-progress level, the utility of imagined futures varies across execution stages as control demands change; and (ii) at the intra-progress level, individual future latents exhibit heterogeneous relevance within the same progress state. To address these limitations, we propose ProWAM, a Progress-Conditioned World Action Model that introduces execution progress as an explicit intermediate representation for adaptive imagination utilization. ProWAM comprises two tightly coupled components: (1) To obtain a reliable representation of execution progress, we propose the Self-Supervised Dual-Temporal Progress Encoder (SS-DTPE). SS-DTPE couples short-term action-observation interaction modeling with long-term recurrent progress aggregation to capture recent execution feedback and accumulated task history. (2) Conditioned on the progress representation from SS-DTPE, we propose the Hierarchical Progress-Conditioned Imagination Modulation (HPIM) to adapt imagination utilization to execution progress. HPIM operates at two complementary levels: an inter-progress global modulation mechanism adapts future utilization across execution stages, while an intra-progress relevance mechanism differentiates individual future latents within each progress state. Extensive experiments demonstrate consistent gains over strong VLA and WAM baselines.

Related papers

• Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving [78.83165764484562]
  Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
  arXiv  Detail & Related papers  (2026-09-03T09:21:19Z)
• WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression [24.010807495676605]
  Generative world models provide robots with predictive models of how the world evolves under interaction.<n>We introduce WALL-SS, a world model that generates visual futures through Scale-wise autoregressive Scaling.<n>We show that WALL-SS improves action following and trajectory accuracy, supports coherent minute-long streaming rollout under bounded memory, and consistently benefits from on-policy alignment.
  arXiv  Detail & Related papers  (2026-08-26T17:57:12Z)
• LeapTS: Rethinking Time Series Forecasting as Adaptive Multi-Horizon Scheduling [74.94985663101906]
  We present LeapTS, a novel framework that reformulates time series forecasting as a dynamic scheduling process over the prediction horizon.<n>We show that LeapTS improves overall forecasting performance by at least 7.4% while achieving a 2.6$times$ to 5.3$times$ inference speedup over representative Transformer-based models.
  arXiv  Detail & Related papers  (2026-05-11T09:54:02Z)
• ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulation [57.07494675832939]
  Most existing vision-language-action (VLA) models for robotic manipulation lack progress awareness.<n>In this work, we investigate the estimation and integration of task progress, proposing a novel model named textbf vla<n>Experiments on the CALVIN and LIBERO benchmarks, alongside real-world robot deployment, consistently demonstrate substantial improvements in success rates.
  arXiv  Detail & Related papers  (2026-03-29T12:38:11Z)
• FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment [45.21358158991569]
  VLA models can predict environmental dynamics, known as world modeling.<n>Current approaches face two main issues: 1) The training objective forces models to over-emphasize pixel-level reconstruction, which constrains semantic learning and generalization, and 2) Reliance on predicted future observations during inference often leads to error accumulation.<n>We introduce Future Representation Alignment via Parallel Progressive Expansion (FRAPPE) to address these challenges.
  arXiv  Detail & Related papers  (2026-02-19T11:00:46Z)
• FUTURE-VLA: Forecasting Unified Trajectories Under Real-time Execution [27.399648455932397]
  FUTURE-VLA is a unified architecture that reformulates long-horizon control and future forecasting as a monolithic sequence-generation task.<n> FUTURE-VLA achieves success rates of 99.2% on LIBERO, 75.4% on RoboTwin, and 78.0% on a real-world Piper platform.
  arXiv  Detail & Related papers  (2026-02-05T14:27:43Z)
• ProgD: Progressive Multi-scale Decoding with Dynamic Graphs for Joint Multi-agent Motion Forecasting [19.47641314660009]
  Accurate motion prediction of surrounding agents is crucial for the safe planning of autonomous vehicles.<n>Recent advancements have extended prediction techniques from individual agents to joint predictions of interacting multiple agents.<n>We propose a progressive multi-scale decoding strategy, termed ProgD, with the help of dynamic heterogeneous graph-based scenario modeling.
  arXiv  Detail & Related papers  (2025-09-11T07:36:54Z)
• Ego-centric Predictive Model Conditioned on Hand Trajectories [52.531681772560724]
  In egocentric scenarios, anticipating both the next action and its visual outcome is essential for understanding human-object interactions.<n>We propose a unified two-stage predictive framework that jointly models action and visual future in egocentric scenarios.<n>Our approach is the first unified model designed to handle both egocentric human activity understanding and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-08-27T13:09:55Z)
• HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model [54.64088247291416]
  A fundamental objective of manipulation policy design is to endow robots to comprehend human instructions, reason about scene cues, and execute generalized actions in dynamic environments.<n>Recent autoregressive vision-language-action (VLA) methods inherit common-sense reasoning capabilities from vision-language models (VLMs) for next action-token prediction.<n>We introduce HybridVLA, a unified framework that absorbs the continuous nature of diffusion-based actions and the contextual reasoning of autoregression.
  arXiv  Detail & Related papers  (2025-03-13T17:59:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.