H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model

• URL: http://arxiv.org/abs/2602.11291v2
• Date: Wed, 04 Mar 2026 17:05:48 GMT
• Title: H-WM: Robotic Task and Motion Planning Guided by Hierarchical World Model
• Abstract summary: We propose a Hierarchical World Model (H-WM) that jointly predicts logical and visual state transitions within a unified framework.<n>H-WM combines a high-level logical world model with a low-level visual world model, integrating the long-horizon of symbolic reasoning with visual grounding.<n> Experiments across multiple vision-language-action (VLA) control policies demonstrate the effectiveness and generality of H-WM's guidance.
• Score: 26.305515799597714
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: World models are becoming central to robotic planning and control as they enable prediction of future state transitions. Existing approaches often emphasize video generation or natural-language prediction, which are difficult to ground in robot actions and suffer from compounding errors over long horizons. Classic task and motion planning models world transitions in logical space, enabling robot-executable and robust long-horizon reasoning. However, they typically operate independently of visual perception, preventing synchronized symbolic and visual state prediction. We propose a Hierarchical World Model (H-WM) that jointly predicts logical and visual state transitions within a unified framework. H-WM combines a high-level logical world model with a low-level visual world model, integrating the long-horizon robustness of symbolic reasoning with visual grounding. The hierarchical outputs provide stable intermediate guidance for long-horizon tasks, mitigating error accumulation and enabling robust execution across extended task sequences. Experiments across multiple vision-language-action (VLA) control policies demonstrate the effectiveness and generality of H-WM's guidance.

Related papers

• GWM-VLA: Geometry-Aware Latent World Modeling for Vision-Language-Action Learning [28.596605555817032]
  Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but often degrade under visual and environmental shifts.<n>We propose GWM-VLA, a geometry-aware latent world modeling framework for VLA learning.
  arXiv  Detail & Related papers  (2026-08-07T06:41:51Z)
• NavWM: A Unified Navigation World Model for Foresight-Driven Planning [15.801647085785826]
  NavWM is a unified navigation world model that seamlessly integrates latent world reasoning, multimodal action prediction, and controllable visual generation.<n>To overcome the limitations of deterministic policies, we introduce an anchor-based multimodal trajectory forecasting framework.<n>Experiments across diverse datasets demonstrate that NavWM significantly advances the state-the-art.
  arXiv  Detail & Related papers  (2026-06-23T03:30:20Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving [56.50032398571235]
  GraphWorld is an E2E-AD framework that explicitly enhances long-horizon planning through latent world modeling.<n>We present a World-State-Conditioned Planning that learns ego-centric latent world representations by modeling interactions between an ego vehicle and surrounding agents.<n>This latent world state captures key interaction dynamics and safety-relevant semantics, and serves as a conditioning signal to guide long-horizon, safety-aware trajectory planning.
  arXiv  Detail & Related papers  (2026-06-15T06:26:46Z)
• X-Foresight: A Joint Vision-Action Causal Forecasting Network via Predictive World Modeling [47.54820149491433]
  We introduce X-Foresight, a predictive world model integrated directly into the Vision-Language-Action architecture.<n>By predicting semantically distant chunks rather than adjacent frames, X-Foresight escapes trivial extrapolation.<n> Comprehensive experiments demonstrate that X-Foresight significantly outperforms VLA baselines in planning performance.
  arXiv  Detail & Related papers  (2026-05-24T06:37:04Z)
• LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations [9.35652906982887]
  We propose Least Action World Models (LaWM) to operationalize the Principle of Least Action in learned visual latent space.<n>LaWM encodes observations into learned generalized coordinates, learns a latent discrete Lagrangian over consecutive latent states.<n>Because the transition is induced by a discrete variational principle, LaWM provides a structure-preserving bias for long-horizon visual prediction.
  arXiv  Detail & Related papers  (2026-05-08T07:03:13Z)
• Hierarchical Planning with Latent World Models [49.82095442483551]
  hierarchical planning achieves higher success while requiring up to 4x less planning-time compute.<n>We demonstrate that this hierarchical approach enables zero-shot control on real-world non-greedy robotic tasks.
  arXiv  Detail & Related papers  (2026-04-03T17:32:36Z)
• Act2Goal: From World Model To General Goal-conditioned Policy [14.222177107215648]
  Act2Goal is a goal-conditioned manipulation policy that integrates a goal-conditioned visual world model with multi-scale temporal control.<n>We show that Act2Goal improves success rates from 30% to 90% on challenging out-of-distribution tasks within minutes of autonomous interaction.
  arXiv  Detail & Related papers  (2025-12-29T15:28:42Z)
• PAN: A World Model for General, Interactable, and Long-Horizon World Simulation [49.805071498152536]
  We introduce PAN, a general, interactable, and long-horizon world model.<n>It predicts future world states through high-quality video simulation conditioned on history and natural language actions.<n>Experiments show that PAN achieves strong performance in action-conditioned world simulation, long-horizon forecasting, and simulative reasoning.
  arXiv  Detail & Related papers  (2025-11-12T07:20:35Z)
• Unified World Models: Memory-Augmented Planning and Foresight for Visual Navigation [69.94565127141483]
  Current approaches separate navigation planning from visual world modeling, leading to state-action misalignment and limited adaptability.<n>We propose UniWM, a unified, memory-augmented world model integrating egocentric visual foresight and planning within a single multimodal autoregressive backbone.<n>We show that UniWM substantially improves navigation success rates by up to 30%, significantly reduces trajectory errors compared to strong baselines, and exhibits impressive zero-shot generalization on the unseen TartanDrive dataset.
  arXiv  Detail & Related papers  (2025-10-09T18:18:11Z)
• Learning Primitive Embodied World Models: Towards Scalable Robotic Learning [50.32986780156215]
  We propose a novel paradigm for world modeling--Primitive Embodied World Models (PEWM)<n>By restricting video generation to fixed short horizons, our approach enables fine-grained alignment between linguistic concepts and visual representations of robotic actions.<n>Our framework bridges the gap between fine-grained physical interaction and high-level reasoning, paving the way toward scalable, interpretable, and general-purpose embodied intelligence.
  arXiv  Detail & Related papers  (2025-08-28T14:31:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.