Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving

• URL: http://arxiv.org/abs/2609.03572v1
• Date: Thu, 03 Sep 2026 09:21:19 GMT
• Title: Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
• Abstract summary: Drive-HWM is a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales.<n>Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive.
• Score: 78.83165764484562
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World models offer a promising paradigm for autonomous driving by predicting how traffic scenes may evolve and using such predictions to support action generation. However, existing approaches either separate future prediction from action generation or jointly predict them at the same temporal scale, making it difficult to simultaneously achieve long-horizon anticipation and responsive, observation-grounded decision making. We present Drive-HWM, a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales. The slow world model predicts multi-step future representations to capture extended scene evolution. To explicitly model the abundant motion dynamics in driving environments, we introduce Dynamic-Aware Latents learned through optical-flow prediction. Guided by these future representations, the fast model uses a lightweight multimodal backbone and an autoregressive expert to jointly predict the next frame and the immediate action from the latest observation. Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive. Extensive experiments on NAVSIM v1 and v2 demonstrate the strong driving performance of Drive-HWM. Comprehensive ablation studies further validate the effectiveness of the hierarchical slow--fast design, dynamics-aware future representations, and joint next-frame and action prediction.

Related papers

• WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression [24.010807495676605]
  Generative world models provide robots with predictive models of how the world evolves under interaction.<n>We introduce WALL-SS, a world model that generates visual futures through Scale-wise autoregressive Scaling.<n>We show that WALL-SS improves action following and trajectory accuracy, supports coherent minute-long streaming rollout under bounded memory, and consistently benefits from on-policy alignment.
  arXiv  Detail & Related papers  (2026-08-26T17:57:12Z)
• ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving [12.68658021226119]
  We introduce ForgeDrive, a unified autoregressive diffusion framework with visual-action cross-conditioning.<n>We find that ForgeDrive not only unifies driving simulation, planning, and visual odometry into a single model, but also outperforms existing strong planners without any post-training strategy.
  arXiv  Detail & Related papers  (2026-06-30T07:05:48Z)
• IDOL: Inverse-Dynamics-Guided Future Prediction for End-to-End Autonomous Driving [6.515607168928518]
  IDOL is an inverse-dynamics-guided future prediction framework for world-model-based end-to-end planning in latent BEV space.<n>IDOL first predicts multiple future latent scene states with a BEV world model, then applies an inverse dynamics model to adjacent latent futures.<n>These inverse-dynamics-derived signals are used to optimize the planned trajectory, turning future forecasting from passive scene anticipation into actionable planning guidance.
  arXiv  Detail & Related papers  (2026-05-29T16:05:42Z)
• Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation [66.7879424097418]
  We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying vision and motion representation.<n>A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories.<n>Experiments on the NAVSIM, NAVSIM-v2, and nuScenes benchmarks demonstrate that WorldDrive achieves leading planning performance among vision-only methods.
  arXiv  Detail & Related papers  (2026-03-16T07:59:39Z)
• FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Model [103.2513470454204]
  FutureX is a pipeline that enhances end-to-end planners to perform complex motion planning via future scene latent reasoning and trajectory refinement.<n>FutureX enhances existing methods by producing more rational motion plans and fewer collisions without compromising efficiency.
  arXiv  Detail & Related papers  (2025-12-12T02:12:49Z)
• Future-Aware End-to-End Driving: Bidirectional Modeling of Trajectory Planning and Scene Evolution [96.25314747309811]
  We introduce SeerDrive, a novel end-to-end framework that jointly models future scene evolution and trajectory planning.<n>Our method first predicts future bird's-eye view (BEV) representations to anticipate the dynamics of the surrounding scene.<n>Two key components enable this: (1) future-aware planning, which injects predicted BEV features into the trajectory planner, and (2) iterative scene modeling and vehicle planning.
  arXiv  Detail & Related papers  (2025-10-13T07:41:47Z)
• Ego-centric Predictive Model Conditioned on Hand Trajectories [52.531681772560724]
  In egocentric scenarios, anticipating both the next action and its visual outcome is essential for understanding human-object interactions.<n>We propose a unified two-stage predictive framework that jointly models action and visual future in egocentric scenarios.<n>Our approach is the first unified model designed to handle both egocentric human activity understanding and robotic manipulation tasks.
  arXiv  Detail & Related papers  (2025-08-27T13:09:55Z)
• MDMP: Multi-modal Diffusion for supervised Motion Predictions with uncertainty [7.402769693163035]
  This paper introduces a Multi-modal Diffusion model for Motion Prediction (MDMP)<n>It integrates skeletal data and textual descriptions of actions to generate refined long-term motion predictions with quantifiable uncertainty.<n>Our model consistently outperforms existing generative techniques in accurately predicting long-term motions.
  arXiv  Detail & Related papers  (2024-10-04T18:49:00Z)
• State-space Decomposition Model for Video Prediction Considering Long-term Motion Trend [3.910356300831074]
  We propose a state-space decomposition video prediction model that decomposes the overall video frame generation into deterministic appearance prediction and motion prediction. We infer the long-term motion trend from conditional frames to guide the generation of future frames that exhibit high consistency with the conditional frames.
  arXiv  Detail & Related papers  (2024-04-17T17:19:48Z)
• AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving [59.94343412438211]
  We introduce the GPT style next token motion prediction into motion prediction. Different from language data which is composed of homogeneous units -words, the elements in the driving scene could have complex spatial-temporal and semantic relations. We propose to adopt three factorized attention modules with different neighbors for information aggregation and different position encoding styles to capture their relations.
  arXiv  Detail & Related papers  (2024-03-20T06:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.