ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games

• URL: http://arxiv.org/abs/2607.26712v1
• Date: Wed, 29 Jul 2026 09:55:54 GMT
• Title: ActSWM: Action-Sensitive World Models for Long-Horizon Planning in Open-World Games
• Abstract summary: We propose ActSWM, an action-sensitive latent world model grounded in a transition-separation principle.<n>Across step-drift analysis, closed-loop Minecraft planning, and cross-game local action recovery, ActSWM preserves larger action-dependent rollout gaps than existing baselines.
• Score: 6.8149466180197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Latent world models support efficient model-predictive control by optimizing future control sequences in latent space and replanning in a receding-horizon manner. However, existing latent predictors often lack stable long-horizon rollout ability, and prediction accuracy alone does not ensure that rollouts remain responsive to the actions being planned. We identify Context Collapse, a failure mode in which autoregressive latent predictors maintain high similarity to future states while producing nearly indistinguishable futures under different action sequences. To address this issue, we propose ActSWM, an action-sensitive latent world model grounded in a transition-separation principle: a planning-useful latent dynamics model should keep alternative-action futures distinguishable and make the action associated with each local transition recoverable. Under this principle, action sensitivity is enforced as a constraint on latent rollouts rather than treated only as an auxiliary prediction target, encouraging predicted futures to preserve action-dependent differences over long horizons. Across step-drift analysis, closed-loop Minecraft planning, and cross-game local action recovery, ActSWM preserves larger action-dependent rollout gaps than existing baselines, improves task success in long-horizon interactive settings, and enables world-model-based action recovery from offline gameplay videos.

Related papers

• WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression [24.010807495676605]
  Generative world models provide robots with predictive models of how the world evolves under interaction.<n>We introduce WALL-SS, a world model that generates visual futures through Scale-wise autoregressive Scaling.<n>We show that WALL-SS improves action following and trajectory accuracy, supports coherent minute-long streaming rollout under bounded memory, and consistently benefits from on-policy alignment.
  arXiv  Detail & Related papers  (2026-08-26T17:57:12Z)
• SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation [65.81476183180527]
  We introduce SegDiff, a closed-loop visuomotor policy that integrates the strengths of both paradigms.<n>SegDiff decomposes demonstrations into motion segments between keyposes and learns to predict the continuous trajectory from the current state to the next keypose.<n>SegDiff demonstrates significant performance gains over existing approaches across various simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2026-07-13T02:48:03Z)
• ACID: Action Consistency via Inverse Dynamics for Planning with World Models [42.33354357727536]
  ACID is a decision-time planning framework that introduces cycle action consistency.<n>It consistently improves planning and matches the baseline's accuracy with substantially less planning compute.
  arXiv  Detail & Related papers  (2026-07-02T16:38:10Z)
• Beyond the Next Step: Variable-Length Latent World Models for Long-Horizon Planning [40.09669638091686]
  We propose a framework that learns to predict future latent states conditioned on action sequences of variable lengths.<n>Experiments show that Variable-length Latent World Models significantly improve latent space world models.
  arXiv  Detail & Related papers  (2026-06-19T21:50:15Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• CAPE: Contrastive Action-conditioned Parallel Encoding for Embodied Planning [15.559483151991813]
  Embodied agents need to predict the future consequences of candidate actions in order to plan effectively before execution.<n>We propose CAPE, a Contrastive Action-conditioned Parallel Conver framework that learns visual dynamics by distinguishing the future outcomes induced by different action sequences.
  arXiv  Detail & Related papers  (2026-06-05T14:21:44Z)
• IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving [6.515607168928518]
  IDOL is an inverse-dynamics-guided future prediction framework for world-model-based end-to-end planning in latent BEV space.<n>IDOL first predicts multiple future latent scene states with a BEV world model, then applies an inverse dynamics model to adjacent latent futures.<n>These inverse-dynamics-derived signals are used to optimize the planned trajectory, turning future forecasting from passive scene anticipation into actionable planning guidance.
  arXiv  Detail & Related papers  (2026-05-29T16:05:42Z)
• From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation [88.39072412680633]
  We propose MoLA, a control-oriented interface that transforms imagined future videos into executable representations.<n>We evaluate our approach on simulated benchmarks and real-world robot manipulation tasks.
  arXiv  Detail & Related papers  (2026-05-12T14:15:16Z)
• World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems [32.4918638412746]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for building embodied agents that ground perception and language into action.<n>We introduce World-Value-Action (WAV) model, a unified framework that enables implicit planning in VLA systems.
  arXiv  Detail & Related papers  (2026-04-16T07:46:05Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving [52.04950569530877]
  We present Uni-World VLA, a unified vision-language-action model that tightly interleaves future frame prediction and trajectory planning.<n>Our approach achieves competitive closed-loop planning performance while producing high-fidelity future frame predictions.
  arXiv  Detail & Related papers  (2026-03-28T14:39:51Z)
• Self-Correcting VLA: Online Action Refinement via Sparse World Imagination [55.982504915794514]
  We propose Self-Correcting VLA (SC-VLA), which achieve self-improvement by intrinsically guiding action refinement through sparse imagination.<n>SC-VLA achieve state-of-the-art performance, yielding the highest task throughput with 16% fewer steps and a 9% higher success rate than the best-performing baselines.
  arXiv  Detail & Related papers  (2026-02-25T06:58:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.